-- Logs begin at Mon 2022-09-19 11:13:57 UTC, end at Mon 2022-09-19 12:46:23 UTC. -- Sep 19 11:42:38.590377 dv-cinder-858293-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 19 11:42:38.598280 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 19 11:42:38.600275 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 11:42:38 2022] *** Sep 19 11:42:38.600275 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 11:42:38.600275 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 11:42:38.600275 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: nodename: dv-cinder-858293-1-iscsi Sep 19 11:42:38.600275 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: machine: x86_64 Sep 19 11:42:38.600275 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: clock source: unix Sep 19 11:42:38.600275 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: pcre jit disabled Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: detected number of CPU cores: 4 Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: current working directory: / Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: detected binary path: /usr/bin/uwsgi-core Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: your processes number limit is 47680 Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: your memory page size is 4096 bytes Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: detected max file descriptor number: 2048 Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: lock engine: pthread robust mutexes Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: thunder lock: enabled Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 19 11:42:38.600726 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 11:42:38.640681 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: Python main interpreter initialized at 0x56209e6fa9c0 Sep 19 11:42:38.640681 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: python threads support enabled Sep 19 11:42:38.640681 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: your server socket listen backlog is limited to 100 connections Sep 19 11:42:38.640681 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: your mercy for graceful operations on workers is 90 seconds Sep 19 11:42:38.641085 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 11:42:38.641169 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: *** Operational MODE: preforking *** Sep 19 11:42:38.641403 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: *** uWSGI is running in multiple interpreter mode *** Sep 19 11:42:38.641403 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: spawned uWSGI master process (pid: 81392) Sep 19 11:42:38.641610 dv-cinder-858293-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 19 11:42:38.642506 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: spawned uWSGI worker 1 (pid: 81405, cores: 1) Sep 19 11:42:38.642594 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: spawned uWSGI worker 2 (pid: 81406, cores: 1) Sep 19 11:42:38.645807 dv-cinder-858293-1-iscsi devstack@keystone.service[81392]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 11:42:42.965365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:42:42.965365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Sep 19 11:42:42.966222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:42:42.966222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Sep 19 11:42:42.966349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:42:42.966349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: from cryptography.utils import int_from_bytes Sep 19 11:42:42.966765 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:42:42.966765 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: from cryptography.utils import int_from_bytes Sep 19 11:42:43.018905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 11:42:43.019125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 11:42:43.019269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] command line args: [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 11:42:43.019364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 11:42:43.019476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 11:42:43.019564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 11:42:43.019676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] command line args: [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 11:42:43.019764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ================================================================================ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 11:42:43.019935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 11:42:43.020102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ================================================================================ {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 11:42:43.020179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_dir = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] admin_token = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_source = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] config_dir = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] debug = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.020882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] config_source = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] debug = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.021949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_config_append = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] list_limit = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] log_config_append = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_dir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.022968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] log_dir = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] log_file = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023376 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.023991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.024923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025844 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.025979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.026124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.026225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.026505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.026631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.026810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] publish_errors = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.026998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] publish_errors = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.027946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] standard_threads = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] standard_threads = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_journal = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.028993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_json = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_stderr = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] use_journal = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_syslog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] use_json = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] use_stderr = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.029862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.029955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] use_syslog = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.030104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.030190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:42:43.030363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.030449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.030599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.030677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.030846 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.030931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.031888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032516 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.032991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.033187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.033267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.033439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.033520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.033677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.033755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.033948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.034949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.035958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.036900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.037992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.038931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.039916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.040978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.041922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.042942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.043977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.044915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.045000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.045678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.045785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.045935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.046979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.047905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.048846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.049964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.050980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.051928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052844 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.052983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.053994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.054930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.external = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.055883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] auth.external = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.password = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.token = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.056985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] auth.password = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] auth.token = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.057949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.058972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] credential.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.059904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.060931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.061694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.061824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.062884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063762 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.063993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.064985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.065986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.066974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.067879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.068984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.069968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.070939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.071937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.072878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.073955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.074935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.075958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.076111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.076190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.076353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.076520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.076601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.076841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.076937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.077012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.077086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.077222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.077302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.077468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.077556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.077703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.078195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.078283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.078450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.078549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.078678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.078759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.078919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.079879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080774 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.080917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.081968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083637 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.083883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.084967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.085954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.086855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.087980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.088929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.089984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] role.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.090908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] role.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.091864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.092063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.092182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.092301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.092409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.092548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.092680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.092798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.093005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.093087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.093249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.093440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.093522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.093729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.094009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.094098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.094334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.094426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.094684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.094925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.095282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.096209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.096367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.096642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.096954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.097291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.097543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.097845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.098051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.098348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.098631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.099114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.099377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.101378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.101378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.101378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.101697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.101945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.102940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.103993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.104982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.105926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.connection = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.106921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.107026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.107108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.107263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.107354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.107502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.108979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.109956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.110970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.111057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 11:42:43.111133 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.111395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.111749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.112195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.112725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.112999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.113310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:42:43.113492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 11:42:43.141446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.141734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.141988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.142548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.142950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.143040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.150669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.151452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.179730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.179921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.179997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.180075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.236709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.236882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.286212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.286442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.290528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.290766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.444992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.445970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.446097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.446184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.446383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.469953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.470163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.470351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.470467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.470585 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.470673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.470801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.470884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.471019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.471105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.471230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.471312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.471437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.471647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.482403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.482520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.495813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.495989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.496873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.497007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.497092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.497221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.497432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:42:43.513986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.514116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.514221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:42:43.514310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:42:43.514561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.514665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.514766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:42:43.514843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:42:43.515042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.515183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.515323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.515452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.515580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.515707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.515835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.515963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.525265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.525411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.525556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.525687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.525838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.525979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.526113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.526249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.533484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.533629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.536368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.536509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.536637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.536781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.536911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.537045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.537190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.537317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.537448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.537580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.537706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.537873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.538919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.539046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.539176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.562084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.562267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.563557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.563718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.563852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.564111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.568559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.568781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.568884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.569066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.569174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.569353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.575555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.575718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.575840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.576012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.582531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.582709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.582826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.583970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.584160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.584243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.584438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.584536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.584715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.599713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.599993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.600197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.600478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.603781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.604317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.606964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.607260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.607682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.607987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.611411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.612189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.614473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.615388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.617614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.617941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.618230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.618512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.618660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.618788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.618961 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.619072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.619248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.619537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.619829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.620128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.635069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.635354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.635624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.635889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.636460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.636752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.637021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.637286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.643299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.644790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.644929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.646314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.646410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.646590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.647858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.648136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.649328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.649613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.649928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.650214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.650597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.650833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.651121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.651408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.651696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.652097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.661628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.661938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.662224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.662505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.662883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.663175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.663260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.663455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.663539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.663807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.664092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.664471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.664764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.665045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.678673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.680400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.681538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.683320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.684436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.686413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.686745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.687875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.688157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.688246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.688549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.689735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.690122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.690529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.694875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.696335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.696806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.698274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.699225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.699625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.699913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.700195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.701137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.701578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.701894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.702181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.708327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.708614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.708897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.710219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.710515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.710800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.715753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.716039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.716322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.716607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.716873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.717679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.717980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.718199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:42:43.718298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.718493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.718591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.718777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.718874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.719059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.719153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.723618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.723738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.723827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.723908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.723908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.724446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:42:43.755255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56209e6fa9c0 pid: 81406 (default app) Sep 19 11:42:43.756790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56209e6fa9c0 pid: 81405 (default app) Sep 19 11:42:43.758656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f782c01-e571-4fb6-802f-628d1a56a57a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:43.758943 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f782c01-e571-4fb6-802f-628d1a56a57a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:43.759216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f782c01-e571-4fb6-802f-628d1a56a57a None None] PATH_INFO: `/v3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:43.763373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Mon Sep 19 11:42:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:42:50.767580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e209af5-5195-4b6d-9e3c-e1af9995413f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:50.768101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e209af5-5195-4b6d-9e3c-e1af9995413f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:50.768101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e209af5-5195-4b6d-9e3c-e1af9995413f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:50.772343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1/2] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:42:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:42:50.777845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:50.778320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:50.778460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:50.847886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: INFO dbcounter [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] Registered counter for database keystone Sep 19 11:42:50.867775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG oslo_db.sqlalchemy.engines [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 11:42:50.874970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writer thread running {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 11:42:51.055976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81406) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 11:42:51.056275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81406) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 11:42:51.116040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:42:51.117653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:42:51.252880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 2/3] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:42:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 19 11:42:51.703913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e209af5-5195-4b6d-9e3c-e1af9995413f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:51.773732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: INFO dbcounter [None req-9e209af5-5195-4b6d-9e3c-e1af9995413f None None] Registered counter for database keystone Sep 19 11:42:51.795293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG oslo_db.sqlalchemy.engines [None req-9e209af5-5195-4b6d-9e3c-e1af9995413f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81405) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 11:42:51.801876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writer thread running {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 11:42:51.925331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e209af5-5195-4b6d-9e3c-e1af9995413f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:51.929671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77994f-3216-4cd5-b5d1-01941c78fce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:51.930346 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77994f-3216-4cd5-b5d1-01941c78fce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:51.930625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77994f-3216-4cd5-b5d1-01941c78fce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:51.930907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77994f-3216-4cd5-b5d1-01941c78fce7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:51.941292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77994f-3216-4cd5-b5d1-01941c78fce7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:51.992693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING keystone.common.rbac_enforcer.enforcer [None req-df77994f-3216-4cd5-b5d1-01941c78fce7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 11:42:51.992949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77994f-3216-4cd5-b5d1-01941c78fce7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:52.004123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 2/4] 10.222.0.88 () {58 vars in 1324 bytes} [Mon Sep 19 11:42:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:42:52.011280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:52.028993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9296f-3adc-4150-a91f-f63922912bcf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:52.033494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca1042-a182-4861-bc55-870416307245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:52.034683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca1042-a182-4861-bc55-870416307245 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:52.034965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca1042-a182-4861-bc55-870416307245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:52.035237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca1042-a182-4861-bc55-870416307245 admin admin] PATH_INFO: `/v3/projects/a73ceb365cbe45598f5feb33f3998086` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:52.036900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca1042-a182-4861-bc55-870416307245 admin admin] RBAC: Authorizing `identity:get_project(project_id=a73ceb365cbe45598f5feb33f3998086)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:52.108661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9aca1042-a182-4861-bc55-870416307245 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 11:42:52.110848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca1042-a182-4861-bc55-870416307245 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:52.113986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 3/5] 10.222.0.88 () {56 vars in 1207 bytes} [Mon Sep 19 11:42:52 2022] GET /identity/v3/projects/a73ceb365cbe45598f5feb33f3998086 => generated 344 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:42:54.305045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c7535-776f-4431-8f78-5d6dcffcbc69 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:54.305580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c7535-776f-4431-8f78-5d6dcffcbc69 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:54.305580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c7535-776f-4431-8f78-5d6dcffcbc69 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:54.309974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 3/6] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:42:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:42:54.316566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd79e4-44da-42b0-aeb2-3295115182e4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:54.317087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd79e4-44da-42b0-aeb2-3295115182e4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:54.317174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd79e4-44da-42b0-aeb2-3295115182e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:54.341832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-77cd79e4-44da-42b0-aeb2-3295115182e4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:42:54.343975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77cd79e4-44da-42b0-aeb2-3295115182e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:42:54.352395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 4/7] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:42:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 19 11:42:54.626791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c7535-776f-4431-8f78-5d6dcffcbc69 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:54.639759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c7535-776f-4431-8f78-5d6dcffcbc69 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:54.644121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfe07-22ee-48ac-b2d7-5d8bcc3cf63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:54.644735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfe07-22ee-48ac-b2d7-5d8bcc3cf63c admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:54.645016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfe07-22ee-48ac-b2d7-5d8bcc3cf63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:54.645293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfe07-22ee-48ac-b2d7-5d8bcc3cf63c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:54.653754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cfe07-22ee-48ac-b2d7-5d8bcc3cf63c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:54.656133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cfe07-22ee-48ac-b2d7-5d8bcc3cf63c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:54.665546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 4/8] 10.222.0.88 () {58 vars in 1324 bytes} [Mon Sep 19 11:42:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:42:54.671412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd79e4-44da-42b0-aeb2-3295115182e4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:54.680821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd79e4-44da-42b0-aeb2-3295115182e4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:54.685187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d2a39-44ed-4aaf-8b37-76bd550c6cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:54.686349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d2a39-44ed-4aaf-8b37-76bd550c6cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:54.686631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d2a39-44ed-4aaf-8b37-76bd550c6cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:54.686904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d2a39-44ed-4aaf-8b37-76bd550c6cf5 admin admin] PATH_INFO: `/v3/users/e4f47c952a6a40bf944493656ebcef14` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:54.688243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d2a39-44ed-4aaf-8b37-76bd550c6cf5 admin admin] RBAC: Authorizing `identity:get_user(user_id=e4f47c952a6a40bf944493656ebcef14)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:54.690117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d2a39-44ed-4aaf-8b37-76bd550c6cf5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:54.691700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 5/9] 10.222.0.88 () {56 vars in 1198 bytes} [Mon Sep 19 11:42:54 2022] GET /identity/v3/users/e4f47c952a6a40bf944493656ebcef14 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:42:57.063917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef20649-1d13-4fab-b6a3-9f6a72fa464e None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:57.064465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef20649-1d13-4fab-b6a3-9f6a72fa464e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:57.064741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef20649-1d13-4fab-b6a3-9f6a72fa464e None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:57.066444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 5/10] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:42:57.075230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25350b-6dbf-4309-a0aa-f49433ca17ea None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:57.075749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25350b-6dbf-4309-a0aa-f49433ca17ea None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:57.076757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25350b-6dbf-4309-a0aa-f49433ca17ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:57.101607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dc25350b-6dbf-4309-a0aa-f49433ca17ea None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:42:57.102912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc25350b-6dbf-4309-a0aa-f49433ca17ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:42:57.209824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 6/11] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:42:57.230666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb95d36-df4a-44f9-b35a-05f747ae7615 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:57.231049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb95d36-df4a-44f9-b35a-05f747ae7615 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:57.231405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb95d36-df4a-44f9-b35a-05f747ae7615 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:57.232918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 6/12] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:42:57.240904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b557b-f34c-4b32-9f08-4c2a85268f74 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:57.241216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b557b-f34c-4b32-9f08-4c2a85268f74 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:57.241497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b557b-f34c-4b32-9f08-4c2a85268f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:57.263377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9a7b557b-f34c-4b32-9f08-4c2a85268f74 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:42:57.265435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9a7b557b-f34c-4b32-9f08-4c2a85268f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:42:57.289200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:57.352565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 7/13] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:42:57.506371 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb95d36-df4a-44f9-b35a-05f747ae7615 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:57.628270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b557b-f34c-4b32-9f08-4c2a85268f74 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:57.673292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb95d36-df4a-44f9-b35a-05f747ae7615 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:57.716514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:57.717219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:57.717505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:57.717801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:57.762429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:57.777713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b557b-f34c-4b32-9f08-4c2a85268f74 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:57.792952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:57.794548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:57.814616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:57.815261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:57.815543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:57.815819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:57.828622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:42:57.837489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:42:57.838576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:42:57.840008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 8/14] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:42:57.844568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:57.860738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:57.908904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 7/15] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:42:57.914847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:57.952981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525ee04-d76c-4b0d-a4e1-359db54fd781 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:57.988802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:57.989418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:57.989700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:57.989997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:57.990682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:42:57.990983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:57.992418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:58.001478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:58.005682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 9/16] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:42:58.016174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:58.029802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181467d-838b-40e7-b9fc-1a1e1668c330 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:58.075581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:58.076994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:58.077428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:58.077824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:58.099366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:58.100851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:58.108717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:42:58.121509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:42:58.122673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:42:58.124088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:42:58.124088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:42:58.124088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:42:58.124088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 8/17] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:42:58.129825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:58.152609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f972a1-f57a-42ab-b480-7b6bc87921c7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:58.197238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:58.198201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:58.198607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:58.198997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] PATH_INFO: `/v3/users/e4f47c952a6a40bf944493656ebcef14` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:58.201151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] RBAC: Authorizing `identity:get_user(user_id=e4f47c952a6a40bf944493656ebcef14)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:58.203961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:58.206856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 10/18] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:42:58 2022] GET /identity/v3/users/e4f47c952a6a40bf944493656ebcef14 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:42:58.213812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:58.247227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c619-6599-476c-97fa-008bb97a0bea None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:58.279843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:58.289309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:58.289966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:58.290252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:58.290526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:58.291203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:42:58.291510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:58.292934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:58.313818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 9/19] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:42:58.364895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d2ba-6a15-40b8-ba08-3b5a21d881f9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:58.407772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455dbf7-4c0e-48bb-9f90-a27efdd73eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:58.408468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455dbf7-4c0e-48bb-9f90-a27efdd73eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:58.408748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455dbf7-4c0e-48bb-9f90-a27efdd73eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:58.409028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455dbf7-4c0e-48bb-9f90-a27efdd73eb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:58.410397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455dbf7-4c0e-48bb-9f90-a27efdd73eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:58.411922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455dbf7-4c0e-48bb-9f90-a27efdd73eb4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:58.413696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 11/20] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:42:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:42:58.420849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:58.547490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5661b-8c4b-44ba-8ecb-f6aa8889b838 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:58.590233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effa53f-05f4-414f-970a-e920d0b22c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:58.590996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effa53f-05f4-414f-970a-e920d0b22c73 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:58.591375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effa53f-05f4-414f-970a-e920d0b22c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:58.592313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effa53f-05f4-414f-970a-e920d0b22c73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:58.593160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effa53f-05f4-414f-970a-e920d0b22c73 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.domain.id=default, user.id=e4f47c952a6a40bf944493656ebcef14) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:42:58.593627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effa53f-05f4-414f-970a-e920d0b22c73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:58.595416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effa53f-05f4-414f-970a-e920d0b22c73 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:58.605168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 10/21] 10.222.0.88 () {56 vars in 1322 bytes} [Mon Sep 19 11:42:58 2022] GET /identity/v3/role_assignments?user.id=e4f47c952a6a40bf944493656ebcef14&scope.domain.id=default&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 233 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:01.290267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd173-4850-45fc-96e9-17f93294256c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:01.291328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd173-4850-45fc-96e9-17f93294256c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:01.291851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd173-4850-45fc-96e9-17f93294256c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:01.293510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 12/22] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:01.299627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ade28-05bd-4b5f-9d37-496d5417e5a0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:01.300615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ade28-05bd-4b5f-9d37-496d5417e5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:01.301163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ade28-05bd-4b5f-9d37-496d5417e5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:01.324817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG passlib.handlers.bcrypt [None req-dc0ade28-05bd-4b5f-9d37-496d5417e5a0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81405) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 11:43:01.325530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG passlib.handlers.bcrypt [None req-dc0ade28-05bd-4b5f-9d37-496d5417e5a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81405) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 11:43:01.392311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-dc0ade28-05bd-4b5f-9d37-496d5417e5a0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:01.394352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-dc0ade28-05bd-4b5f-9d37-496d5417e5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:01.492903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:01.520661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 11/23] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:01.897038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a28434-66b1-4ec5-8ecd-a71bb7f6c054 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:01.898109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a28434-66b1-4ec5-8ecd-a71bb7f6c054 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:01.898931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a28434-66b1-4ec5-8ecd-a71bb7f6c054 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:01.901869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 13/24] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:01.909446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476fb39-2a4b-49e4-9c1a-9c3cedca46b8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:01.910474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476fb39-2a4b-49e4-9c1a-9c3cedca46b8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:01.911271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476fb39-2a4b-49e4-9c1a-9c3cedca46b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:01.937318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0476fb39-2a4b-49e4-9c1a-9c3cedca46b8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:01.938589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0476fb39-2a4b-49e4-9c1a-9c3cedca46b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:02.001795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6d175-4698-4237-a4fd-6e0fe81c2d9f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:02.002703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6d175-4698-4237-a4fd-6e0fe81c2d9f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:02.003397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6d175-4698-4237-a4fd-6e0fe81c2d9f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:02.010213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 14/25] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:02.017998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba881-fbda-45c4-8052-1caca4ba2b90 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:02.018923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba881-fbda-45c4-8052-1caca4ba2b90 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:02.019482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba881-fbda-45c4-8052-1caca4ba2b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:02.053720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 12/26] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:02.059877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1a2ba881-fbda-45c4-8052-1caca4ba2b90 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:02.061164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1a2ba881-fbda-45c4-8052-1caca4ba2b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:02.070258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:02.188928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 15/27] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:02.212564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476fb39-2a4b-49e4-9c1a-9c3cedca46b8 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:02.384018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476fb39-2a4b-49e4-9c1a-9c3cedca46b8 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:02.407906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:02.444645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:02.445250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:02.445539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:02.445836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:02.463004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:02.464588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:43:02.492385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:02.493393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:43:02.494795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:43:02.494795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:43:02.494795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:02.494795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 13/28] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:02.502149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ba881-fbda-45c4-8052-1caca4ba2b90 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:02.581831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:02.671628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ba881-fbda-45c4-8052-1caca4ba2b90 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:02.729230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435d50-b3bb-47f3-9b09-3025e0644aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:02.729808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435d50-b3bb-47f3-9b09-3025e0644aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:02.730087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435d50-b3bb-47f3-9b09-3025e0644aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:02.730359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435d50-b3bb-47f3-9b09-3025e0644aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:02.731027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435d50-b3bb-47f3-9b09-3025e0644aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:02.731337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435d50-b3bb-47f3-9b09-3025e0644aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:02.734729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435d50-b3bb-47f3-9b09-3025e0644aa0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:02.755307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:02.765960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 16/29] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:02.767770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff962-e8ce-4589-808f-a3dc860fac48 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:02.768280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff962-e8ce-4589-808f-a3dc860fac48 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:02.768817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff962-e8ce-4589-808f-a3dc860fac48 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:02.770610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 17/30] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:02.771651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff962-e8ce-4589-808f-a3dc860fac48 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:02.803746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c2ebc-2258-49c9-bba1-e3e9a34ed06d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:02.853254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:02.854314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:02.854995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:02.855474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:02.875058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:02.882510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:02.883464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:02.884253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 14/31] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:02.885823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:02.922902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:02.943965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff962-e8ce-4589-808f-a3dc860fac48 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:03.016258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079877b-63a4-4cc4-86c8-0bfb8a54587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:03.017318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079877b-63a4-4cc4-86c8-0bfb8a54587d None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.017702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079877b-63a4-4cc4-86c8-0bfb8a54587d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.018102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079877b-63a4-4cc4-86c8-0bfb8a54587d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.018996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079877b-63a4-4cc4-86c8-0bfb8a54587d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:03.020874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079877b-63a4-4cc4-86c8-0bfb8a54587d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:03.031269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978345d-6d86-4c46-b6e0-80cf00577f10 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:03.087367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:03.088272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.088751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.089272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.090759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:03.093168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:03.162709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 18/32] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Sep 19 11:43:02 2022] POST /identity/v3/roles => generated 219 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:03.162709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ddf1-76bd-4fa7-b6bf-324f8a15c9ed None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.162709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ddf1-76bd-4fa7-b6bf-324f8a15c9ed None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.162709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ddf1-76bd-4fa7-b6bf-324f8a15c9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.180536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b179ddf1-76bd-4fa7-b6bf-324f8a15c9ed None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:03.182190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b179ddf1-76bd-4fa7-b6bf-324f8a15c9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:03.251928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 15/33] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:43:02 2022] POST /identity/v3/projects => generated 301 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:03.253056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:03.255881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:03.279733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 19/34] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:03.281589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb151f5-7a5a-4084-88c7-64e2341fa522 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.282186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb151f5-7a5a-4084-88c7-64e2341fa522 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.282665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb151f5-7a5a-4084-88c7-64e2341fa522 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.284476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 20/35] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:03.286593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e81185-bdcb-4f20-8591-32ad35b14590 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.287124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e81185-bdcb-4f20-8591-32ad35b14590 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.287583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e81185-bdcb-4f20-8591-32ad35b14590 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.294449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 21/36] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:03.297867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba47e0-ad33-417f-86fe-97e29f547807 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.298623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba47e0-ad33-417f-86fe-97e29f547807 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.299098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba47e0-ad33-417f-86fe-97e29f547807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.345683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bdba47e0-ad33-417f-86fe-97e29f547807 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:03.349453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bdba47e0-ad33-417f-86fe-97e29f547807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:03.437355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:03.487172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 22/37] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:03.489479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890497b9-f55e-4c7c-847a-40df19ee2792 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.490138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890497b9-f55e-4c7c-847a-40df19ee2792 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.490727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890497b9-f55e-4c7c-847a-40df19ee2792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.491060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943b5c0-da5f-4f38-89c6-ef37d13081bd None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:03.519876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-890497b9-f55e-4c7c-847a-40df19ee2792 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:03.522899 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-890497b9-f55e-4c7c-847a-40df19ee2792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:03.542598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:03.543031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.543784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.543784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.544350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:03.545205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:03.547316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:03.561947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 16/38] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:03.567790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:03.652530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 23/39] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:03.685822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890497b9-f55e-4c7c-847a-40df19ee2792 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:03.746664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:03.772986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49666d34-f8af-4fd1-88fe-7fcf74000b79 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:03.834397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890497b9-f55e-4c7c-847a-40df19ee2792 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:03.835638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:03.836441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.836822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.837196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] PATH_INFO: `/v3/users/e4f47c952a6a40bf944493656ebcef14` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.839084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] RBAC: Authorizing `identity:get_user(user_id=e4f47c952a6a40bf944493656ebcef14)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:03.842062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:03.843950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 17/40] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:03 2022] GET /identity/v3/users/e4f47c952a6a40bf944493656ebcef14 => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:03.844638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:03.850110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:03.888683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:03.889533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:03.889868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:03.890144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:03.890885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:03.892611 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:03.914523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 24/41] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Sep 19 11:43:03 2022] POST /identity/v3/roles => generated 225 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:43:03.914523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:04.003590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd532b3f-5c3c-4cc1-bd3f-df7968e814e5 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:04.056936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:04.057921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:04.058247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:04.058634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:04.060396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:04.062624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:04.064995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 18/42] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:04.068372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:04.085346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c4c37-367f-4077-b0e6-8f85724bd2d8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:04.098820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:04.132863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:04.134711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:04.134711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:04.134897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:04.136147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:04.138087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:04.141289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 25/43] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:04.141289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:04.185992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:04.334948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733931c4-e299-431f-bba6-5d7243d20bdc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:04.375316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:04.375954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:04.376344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:04.376624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:04.377318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:04.378766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:04.380878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f20ee9-1a1e-4696-9821-f573316378b3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:04.395967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 19/44] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Sep 19 11:43:04 2022] POST /identity/v3/roles => generated 223 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:04.396803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:04.425111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367e486-62a5-4a5c-8b76-bbb63013337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:04.425781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367e486-62a5-4a5c-8b76-bbb63013337f None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:04.426077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367e486-62a5-4a5c-8b76-bbb63013337f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:04.426356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367e486-62a5-4a5c-8b76-bbb63013337f None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:04.427063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367e486-62a5-4a5c-8b76-bbb63013337f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:04.428598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367e486-62a5-4a5c-8b76-bbb63013337f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:04.469137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 26/45] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:43:04 2022] POST /identity/v3/projects => generated 312 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:04.543595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acd72c-a4ad-4154-b13b-ba1f6b602bbc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:04.594965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:04.603302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d3101-a99e-4ee6-a966-914ad78fa4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:04.604529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3101-a99e-4ee6-a966-914ad78fa4ad None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:04.604816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3101-a99e-4ee6-a966-914ad78fa4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:04.605813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3101-a99e-4ee6-a966-914ad78fa4ad None admin] PATH_INFO: `/v3/domains/default/users/e4f47c952a6a40bf944493656ebcef14/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:04.609427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d3101-a99e-4ee6-a966-914ad78fa4ad None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e4f47c952a6a40bf944493656ebcef14, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:04.609876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d3101-a99e-4ee6-a966-914ad78fa4ad None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:04.630797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 20/46] 10.222.0.88 () {58 vars in 1361 bytes} [Mon Sep 19 11:43:04 2022] PUT /identity/v3/domains/default/users/e4f47c952a6a40bf944493656ebcef14/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:06.792828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99cb2b-c4a8-4234-8a8e-7e65d963b070 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:06.793395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99cb2b-c4a8-4234-8a8e-7e65d963b070 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:06.793522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99cb2b-c4a8-4234-8a8e-7e65d963b070 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:06.795588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 27/47] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:06.804372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e78caf-bd01-46f0-aeab-c55de3588575 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:06.804983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e78caf-bd01-46f0-aeab-c55de3588575 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:06.805322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e78caf-bd01-46f0-aeab-c55de3588575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:06.833517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a3e78caf-bd01-46f0-aeab-c55de3588575 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:06.836132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a3e78caf-bd01-46f0-aeab-c55de3588575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:06.923727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 21/48] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:07.152122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8c80-e174-4165-9ff0-38cc325e5aaf None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:07.152404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8c80-e174-4165-9ff0-38cc325e5aaf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:07.152683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8c80-e174-4165-9ff0-38cc325e5aaf None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:07.154413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 28/49] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:07.161194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a3985-1a86-4cc8-a1a7-66e7431880e7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:07.161694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a3985-1a86-4cc8-a1a7-66e7431880e7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:07.162235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a3985-1a86-4cc8-a1a7-66e7431880e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:07.191423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2d8a3985-1a86-4cc8-a1a7-66e7431880e7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:07.193544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2d8a3985-1a86-4cc8-a1a7-66e7431880e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:07.204913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8c80-e174-4165-9ff0-38cc325e5aaf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:07.241608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:07.291890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 22/50] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:07.348927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8c80-e174-4165-9ff0-38cc325e5aaf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:07.390587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:07.391215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:07.391704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:07.391984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:07.393205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:07.394758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:07.396428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 29/51] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:07.404575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a3985-1a86-4cc8-a1a7-66e7431880e7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:07.473639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:07.534631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a3985-1a86-4cc8-a1a7-66e7431880e7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:07.585565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:07.593010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:07.593656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:07.593968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:07.594245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:07.594931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:07.596441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:07.642562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 23/52] 10.222.0.88 () {60 vars in 1140 bytes} [Mon Sep 19 11:43:07 2022] POST /identity/v3/projects => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:43:07.731457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:07.737111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6b359-302d-49b0-a85c-12721cc6382f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:07.777702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:07.778391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:07.778670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:07.779437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:07.791518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:07.797729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:07.799158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:07.799964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 30/53] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:07.805352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:07.972338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b4c7c-2e73-49a9-80ca-c6cfb1d2ec1d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:07.975930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:08.020727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:08.021364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:08.021647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:08.021970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:08.022684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:08.022999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:08.024439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:08.036317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 24/54] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:08.042228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:08.165460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528558-8f3a-4545-a299-67d2bdbed1fe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:08.204104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:08.204753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:08.205039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:08.205316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] PATH_INFO: `/v3/users/e4f47c952a6a40bf944493656ebcef14` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:08.206723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] RBAC: Authorizing `identity:get_user(user_id=e4f47c952a6a40bf944493656ebcef14)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:08.208201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:08.209840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 31/55] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:08 2022] GET /identity/v3/users/e4f47c952a6a40bf944493656ebcef14 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:08.215379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:08.347378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a5f28-a6ba-40ba-bfa5-70b03c81cd06 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:08.384528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834aaf2f-7274-4414-ad9c-1445257b4d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:08.385184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834aaf2f-7274-4414-ad9c-1445257b4d92 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:08.385468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834aaf2f-7274-4414-ad9c-1445257b4d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:08.385748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834aaf2f-7274-4414-ad9c-1445257b4d92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:08.387009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834aaf2f-7274-4414-ad9c-1445257b4d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:08.388543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834aaf2f-7274-4414-ad9c-1445257b4d92 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:08.390346 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 25/56] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:08.396387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:08.427680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:08.508510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17709f3-d41b-41c1-961b-2a3c0ed4cac1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:08.550368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201acd8-c699-4e07-acc1-85b68dca7f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:08.550998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201acd8-c699-4e07-acc1-85b68dca7f4b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:08.551278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201acd8-c699-4e07-acc1-85b68dca7f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:08.551546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201acd8-c699-4e07-acc1-85b68dca7f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:08.554203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201acd8-c699-4e07-acc1-85b68dca7f4b None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.domain.id=default, user.id=e4f47c952a6a40bf944493656ebcef14) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:08.554203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201acd8-c699-4e07-acc1-85b68dca7f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:08.554203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201acd8-c699-4e07-acc1-85b68dca7f4b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:08.562186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 32/57] 10.222.0.88 () {56 vars in 1322 bytes} [Mon Sep 19 11:43:08 2022] GET /identity/v3/role_assignments?user.id=e4f47c952a6a40bf944493656ebcef14&scope.domain.id=default&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 530 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:09.903664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b33365-cbda-47eb-9517-44efce5448e7 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:09.904247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b33365-cbda-47eb-9517-44efce5448e7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:09.904334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b33365-cbda-47eb-9517-44efce5448e7 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:09.906058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 26/58] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:09.912041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a9df-e44c-45e5-91c5-f25f3fc41e15 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:09.912534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a9df-e44c-45e5-91c5-f25f3fc41e15 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:09.912618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a9df-e44c-45e5-91c5-f25f3fc41e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:09.931015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-48d0a9df-e44c-45e5-91c5-f25f3fc41e15 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:09.932261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-48d0a9df-e44c-45e5-91c5-f25f3fc41e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:10.004071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 33/59] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:10.282634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b33365-cbda-47eb-9517-44efce5448e7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:10.322842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:10.430626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b33365-cbda-47eb-9517-44efce5448e7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:10.469353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7b608-c985-4bad-955c-2691a2cbb092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:10.470033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7b608-c985-4bad-955c-2691a2cbb092 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:10.470368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7b608-c985-4bad-955c-2691a2cbb092 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:10.470636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7b608-c985-4bad-955c-2691a2cbb092 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:10.471802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7b608-c985-4bad-955c-2691a2cbb092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:10.473253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7b608-c985-4bad-955c-2691a2cbb092 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:10.474911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 27/60] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:10.480665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a9df-e44c-45e5-91c5-f25f3fc41e15 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:10.507435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:10.594252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a9df-e44c-45e5-91c5-f25f3fc41e15 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:10.631289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83510f-53e3-4d74-9f8d-133f4215e8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:10.631910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83510f-53e3-4d74-9f8d-133f4215e8ad None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:10.632192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83510f-53e3-4d74-9f8d-133f4215e8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:10.632461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83510f-53e3-4d74-9f8d-133f4215e8ad None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:10.633127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83510f-53e3-4d74-9f8d-133f4215e8ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:10.634609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83510f-53e3-4d74-9f8d-133f4215e8ad None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:10.664939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 34/61] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:43:10 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:14.394744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f617fbd-ac20-46be-bc2e-d8a2051a06f2 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:14.395521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f617fbd-ac20-46be-bc2e-d8a2051a06f2 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:14.395521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f617fbd-ac20-46be-bc2e-d8a2051a06f2 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:14.396904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 28/62] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:14.403256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9afd4-040b-4292-945a-724d4ae3d766 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:14.404573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9afd4-040b-4292-945a-724d4ae3d766 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:14.404829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9afd4-040b-4292-945a-724d4ae3d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:14.449311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-23d9afd4-040b-4292-945a-724d4ae3d766 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:14.451428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-23d9afd4-040b-4292-945a-724d4ae3d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:14.617668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 35/63] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:14.753138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ca9f3-c19e-40d1-808b-3a03beaf991d None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:14.753599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ca9f3-c19e-40d1-808b-3a03beaf991d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:14.754018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ca9f3-c19e-40d1-808b-3a03beaf991d None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:14.757957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 29/64] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:14.764858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250887d-0dd0-40b5-8b49-296d31f5df6b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:14.765338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250887d-0dd0-40b5-8b49-296d31f5df6b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:14.765862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250887d-0dd0-40b5-8b49-296d31f5df6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:14.795606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:14.804355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f250887d-0dd0-40b5-8b49-296d31f5df6b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:14.811328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f250887d-0dd0-40b5-8b49-296d31f5df6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:14.920057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 36/65] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:14.975753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc814-9935-46b8-99e8-9fdadd600b9f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:14.976695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc814-9935-46b8-99e8-9fdadd600b9f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:14.977478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc814-9935-46b8-99e8-9fdadd600b9f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:14.980223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250887d-0dd0-40b5-8b49-296d31f5df6b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:14.980701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 30/66] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:14.991561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c695e-b33e-425a-bb6a-1bff083dce89 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:14.992516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c695e-b33e-425a-bb6a-1bff083dce89 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:14.993266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c695e-b33e-425a-bb6a-1bff083dce89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.037605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2c7c695e-b33e-425a-bb6a-1bff083dce89 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:15.039625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2c7c695e-b33e-425a-bb6a-1bff083dce89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:15.162443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 31/67] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:15.164075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f1dda-3db1-462d-80c7-70175731c03d None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.164515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f1dda-3db1-462d-80c7-70175731c03d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.164938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f1dda-3db1-462d-80c7-70175731c03d None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.166630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250887d-0dd0-40b5-8b49-296d31f5df6b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.167273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 32/68] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:15.179342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0094e3-025f-4747-a432-9ae5a9abcb84 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.180027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0094e3-025f-4747-a432-9ae5a9abcb84 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.180506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0094e3-025f-4747-a432-9ae5a9abcb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.215963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ca0094e3-025f-4747-a432-9ae5a9abcb84 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:15.217692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ca0094e3-025f-4747-a432-9ae5a9abcb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:15.232230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d43ef-1550-45ce-a413-51f65b543348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:15.232878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d43ef-1550-45ce-a413-51f65b543348 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.233164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d43ef-1550-45ce-a413-51f65b543348 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.233439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d43ef-1550-45ce-a413-51f65b543348 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.238083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:15.251706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d43ef-1550-45ce-a413-51f65b543348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a50d43ef-1550-45ce-a413-51f65b543348 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:15.257332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:15.258472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:15.259260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:15.265239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 37/69] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:15.265811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7790-96aa-4d86-8087-c7f5083c810b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.266101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7790-96aa-4d86-8087-c7f5083c810b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.266379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7790-96aa-4d86-8087-c7f5083c810b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.267641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 38/70] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:15.268539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7790-96aa-4d86-8087-c7f5083c810b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:15.274265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:15.332169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 33/71] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:15.339900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c65c83-c865-455b-a5b4-e4b8147b95dc None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.340469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c65c83-c865-455b-a5b4-e4b8147b95dc None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.341241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c65c83-c865-455b-a5b4-e4b8147b95dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.370624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-10c65c83-c865-455b-a5b4-e4b8147b95dc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:15.372822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-10c65c83-c865-455b-a5b4-e4b8147b95dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:15.391903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7790-96aa-4d86-8087-c7f5083c810b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.428788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:15.429438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.429843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.430392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.431263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:15.431753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:15.433377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:15.444101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 39/72] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:15.445793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:15.467462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 34/73] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:15.468230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c65c83-c865-455b-a5b4-e4b8147b95dc None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:15.518433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:15.563672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840a037-ae54-4017-b64b-bfbcdc1e1b42 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.590914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:15.602645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c65c83-c865-455b-a5b4-e4b8147b95dc None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.603963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:15.605092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.605582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.606090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.618089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:15.624259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:15.625197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 40/74] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:15.630717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:15.639421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:15.640068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.640577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.640681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.658260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:15.659721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:15.661425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 35/75] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:15.662264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:15.742957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3019d-234f-4767-9328-4854df8dadd3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.766636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:15.777921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa940b32-9161-40e3-ae1e-316cf40c8090 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.784119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:15.784932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.785106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.785374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.796174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:15.801821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:15.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:15.804474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 41/76] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:15.805673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:15.825258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:15.826948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.827449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.827891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.848489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:15.854050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:15.858283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 36/77] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:15.859144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:15.873731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:15.937739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb2904-a315-4ef2-adef-05e7c9cc959e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.974670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134b84e-3321-4204-9810-b25253216920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:15.975282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134b84e-3321-4204-9810-b25253216920 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:15.975558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134b84e-3321-4204-9810-b25253216920 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:15.975829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134b84e-3321-4204-9810-b25253216920 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:15.976496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134b84e-3321-4204-9810-b25253216920 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:15.976794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134b84e-3321-4204-9810-b25253216920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:15.978528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134b84e-3321-4204-9810-b25253216920 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:15.979992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4e87e-24ba-4461-a319-a1added0f376 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:15.986763 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 42/78] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:15.988799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134b84e-3321-4204-9810-b25253216920 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.017988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.018613 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.018892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.019169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.034719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.093899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.095675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.097053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134b84e-3321-4204-9810-b25253216920 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.108501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.138716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.139321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.139613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.139883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.141019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.142507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.144101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 43/79] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:16.144891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.209482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 37/80] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:16.210469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.267888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d341c-1021-4c05-81d9-1f6b9f8a88b1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.276990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.303587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.304283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.304481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.304757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.305503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:16.305853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.307227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.311711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d8f6b-d668-4325-9093-2dd6d3e34c36 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.316127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 44/81] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:16.317134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.324970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.353411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.355604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.355604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.355604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.356130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:16.356419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.358415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.367292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 38/82] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:16.368309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.425843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c2899-eb85-4cc8-a702-c317bfeb974f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.460679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.461309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.461598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.461907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] PATH_INFO: `/v3/users/e4f47c952a6a40bf944493656ebcef14` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.463063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] RBAC: Authorizing `identity:get_user(user_id=e4f47c952a6a40bf944493656ebcef14)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.464529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.466861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 45/83] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/users/e4f47c952a6a40bf944493656ebcef14 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:16.467447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.473029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a164e-a179-4b58-8323-8933734bf474 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.508308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.514540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.515161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.515516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.515725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.516381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.517847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.549885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 39/84] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:43:16 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:16.550877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.587375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5124d2-f98f-41ef-8216-76ebd51966b6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.602118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.629717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.630307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.630584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.630863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.632061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.633533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.640304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 46/85] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:16.640304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.698903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c668a-607e-4c0e-a6df-e80cb793c5b1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.741258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.741936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.742219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.742492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.743641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.745089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.746930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 40/86] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:16.748110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.758468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05006e8-0e91-439a-90da-857182272a70 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.769005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.819567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.820606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.821062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.821506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:16.823007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:16.824638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:16.827007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 47/87] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:16.827946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:16.916949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:16.927082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab76b3-a0e9-42cb-9847-0a2feb2e498c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:16.975666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:16.975666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:16.975666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:16.975977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:17.055253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53b341-f026-4ae4-955c-cf026cdbd00e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:17.081321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:17.083932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:17.104614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:17.105248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:17.105530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:17.107473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:17.110066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=d20c9d80453a47e9990b3a60cdaa40ae) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:17.110389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:17.113961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:17.130931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 48/88] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/role_assignments?user.id=d20c9d80453a47e9990b3a60cdaa40ae&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=3530f0bc598248418cc693afa8f4b150 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:17.132993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:17.230728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:17.264425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:17.281174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 41/89] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:17.285664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:17.392627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7cdc5-be88-4e1f-ab83-49ae4d3e8684 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:17.438262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:17.439243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:17.439728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:17.440229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:17.470093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5887c1-afc2-46f8-9820-01576fed93f0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:17.509435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:17.510109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:17.510329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:17.510571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:17.525793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:17.528108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:17.600293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:17.601085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:17.621909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:17.623745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:17.788606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 49/90] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:17.789305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:17.792152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 42/91] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:17.800496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:17.976781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98e5e8-7cc2-426e-8d2b-8553e70d917c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:17.994862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:18.003395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10721f9-5689-41cf-baea-8c275e39c555 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:18.050184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:18.051228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:18.051727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:18.052200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:18.054586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:18.057015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:18.062168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 50/92] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:17 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:18.065496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:18.075260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:18.081693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:18.082248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:18.082730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] PATH_INFO: `/v3/projects/521a1e2f0e87407183cb9476b8214b58` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:18.084173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] RBAC: Authorizing `identity:get_project(project_id=521a1e2f0e87407183cb9476b8214b58)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:18.085924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:18.086524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:18.088813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 43/93] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:17 2022] GET /identity/v3/projects/521a1e2f0e87407183cb9476b8214b58 => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:18.092022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:18.259038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae964f-54a7-4861-b9e7-1b217f5acf75 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:18.300595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:18.301573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:18.302797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:18.303301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:18.304872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=e4f47c952a6a40bf944493656ebcef14) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:18.305557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:18.307653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d575-f9f3-4535-83b5-e6cbafbad341 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:18.309370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:18.324347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 44/94] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:18 2022] GET /identity/v3/role_assignments?user.id=e4f47c952a6a40bf944493656ebcef14&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:18.326093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:18.394156 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:18.394349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:18.395297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:18.395297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:18.395491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:18.399382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:18.400895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:18.402663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 51/95] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:18 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:18.428338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:18.538010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334727-0ad6-44e8-9e98-904ba68a2639 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:18.602227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1e195-b97d-4877-85c5-3105aed8a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:18.602786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1e195-b97d-4877-85c5-3105aed8a746 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:18.603075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1e195-b97d-4877-85c5-3105aed8a746 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:18.603355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1e195-b97d-4877-85c5-3105aed8a746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:18.604350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1e195-b97d-4877-85c5-3105aed8a746 None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=521a1e2f0e87407183cb9476b8214b58, user.id=d20c9d80453a47e9990b3a60cdaa40ae) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:18.604696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1e195-b97d-4877-85c5-3105aed8a746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:18.606327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1e195-b97d-4877-85c5-3105aed8a746 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:18.613010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 45/96] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:18 2022] GET /identity/v3/role_assignments?user.id=d20c9d80453a47e9990b3a60cdaa40ae&scope.project.id=521a1e2f0e87407183cb9476b8214b58&role.id=3530f0bc598248418cc693afa8f4b150 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:18.654627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:18.662157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aeeb5-d636-4a47-bd1a-31cd0a295b9b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:18.713030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c6fa8-8676-4b28-88a9-b5dd7357e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:18.713666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c6fa8-8676-4b28-88a9-b5dd7357e470 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:18.714050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c6fa8-8676-4b28-88a9-b5dd7357e470 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:18.714332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c6fa8-8676-4b28-88a9-b5dd7357e470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:18.715078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c6fa8-8676-4b28-88a9-b5dd7357e470 None admin] RBAC: Adding query filter params (role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=d20c9d80453a47e9990b3a60cdaa40ae) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:18.715394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c6fa8-8676-4b28-88a9-b5dd7357e470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:18.717414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c6fa8-8676-4b28-88a9-b5dd7357e470 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:18.726653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 52/97] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:18 2022] GET /identity/v3/role_assignments?user.id=d20c9d80453a47e9990b3a60cdaa40ae&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=0a108a4463a4458480b00aed843b4e9b => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:19.382168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d416-45a4-4775-8901-69205e5b9f24 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:19.382528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d416-45a4-4775-8901-69205e5b9f24 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:19.382735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d416-45a4-4775-8901-69205e5b9f24 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:19.384248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 46/98] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:19.391294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f63050-0864-4597-aa0c-7058fd2f523c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:19.391806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f63050-0864-4597-aa0c-7058fd2f523c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:19.392240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f63050-0864-4597-aa0c-7058fd2f523c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:19.413555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-93f63050-0864-4597-aa0c-7058fd2f523c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:19.414735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-93f63050-0864-4597-aa0c-7058fd2f523c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:19.512562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 53/99] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:19.719347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce17996-2ae7-49bc-8654-90a40ffb422b None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:19.719831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce17996-2ae7-49bc-8654-90a40ffb422b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:19.719831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce17996-2ae7-49bc-8654-90a40ffb422b None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:19.721461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 47/100] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:19.735700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac261db3-0f5b-454a-a4f0-1dead48ab0eb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:19.736243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac261db3-0f5b-454a-a4f0-1dead48ab0eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:19.736243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac261db3-0f5b-454a-a4f0-1dead48ab0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:19.758864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ac261db3-0f5b-454a-a4f0-1dead48ab0eb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:19.760067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ac261db3-0f5b-454a-a4f0-1dead48ab0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:19.853692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:19.860179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 54/101] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:19.878589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce17996-2ae7-49bc-8654-90a40ffb422b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:20.018161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:20.023011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce17996-2ae7-49bc-8654-90a40ffb422b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:20.059200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:20.059883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:20.060171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:20.060449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:20.071631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:20.077543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:20.078766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:20.079556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 48/102] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:20.086527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac261db3-0f5b-454a-a4f0-1dead48ab0eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:20.214925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:20.242503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac261db3-0f5b-454a-a4f0-1dead48ab0eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:20.322650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:20.323180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:20.323460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:20.323732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:20.324408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:20.324708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:20.326676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:20.349593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 55/103] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:20.358412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:20.427149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986ad38-caf8-44e5-b544-a46b5dbac185 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:20.454956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:20.481065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80704d-269b-44eb-addf-2b98ea92c9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:20.481792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80704d-269b-44eb-addf-2b98ea92c9cc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:20.482085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80704d-269b-44eb-addf-2b98ea92c9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:20.482418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80704d-269b-44eb-addf-2b98ea92c9cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:20.484290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80704d-269b-44eb-addf-2b98ea92c9cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:20.486058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80704d-269b-44eb-addf-2b98ea92c9cc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:20.487949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 49/104] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:20.488963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314130-1055-46bc-9ea9-261b3eaafde5 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:20.489244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314130-1055-46bc-9ea9-261b3eaafde5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:20.493981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314130-1055-46bc-9ea9-261b3eaafde5 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:20.493981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 50/105] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:20.502131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c314130-1055-46bc-9ea9-261b3eaafde5 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:20.514531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25f36c-fd36-4b60-b970-007994c9a50d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:20.543125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:20.556914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778f959-f361-4dce-a386-b7190a6adf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:20.558571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f959-f361-4dce-a386-b7190a6adf44 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:20.559476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f959-f361-4dce-a386-b7190a6adf44 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:20.560363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f959-f361-4dce-a386-b7190a6adf44 None admin] PATH_INFO: `/v3/users/eb70654945034353a5d54f660a61b97c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:20.589448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778f959-f361-4dce-a386-b7190a6adf44 None admin] RBAC: Authorizing `identity:get_user(user_id=eb70654945034353a5d54f660a61b97c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:20.592557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778f959-f361-4dce-a386-b7190a6adf44 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:20.595657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 56/106] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:20 2022] GET /identity/v3/users/eb70654945034353a5d54f660a61b97c => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:20.598539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61de9d-43ab-4819-8403-3a7f664f1b02 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:20.599797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61de9d-43ab-4819-8403-3a7f664f1b02 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:20.601070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61de9d-43ab-4819-8403-3a7f664f1b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:20.637274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bc61de9d-43ab-4819-8403-3a7f664f1b02 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:20.639704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bc61de9d-43ab-4819-8403-3a7f664f1b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:20.705568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c314130-1055-46bc-9ea9-261b3eaafde5 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:20.771086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:20.772832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:20.774796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:20.775270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:20.776668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:20.777127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 57/107] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:20.782487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61de9d-43ab-4819-8403-3a7f664f1b02 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:20.787437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:20.860885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:20.866125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 51/108] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:43:20 2022] POST /identity/v3/projects => generated 302 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:20.959159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61de9d-43ab-4819-8403-3a7f664f1b02 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:21.028974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19500a6-0edd-4ca4-8799-22d73f36bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:21.030107 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19500a6-0edd-4ca4-8799-22d73f36bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:21.030670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19500a6-0edd-4ca4-8799-22d73f36bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:21.031136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19500a6-0edd-4ca4-8799-22d73f36bdc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:21.134198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19500a6-0edd-4ca4-8799-22d73f36bdc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:21.136166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19500a6-0edd-4ca4-8799-22d73f36bdc5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:21.192784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:21.263014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 58/109] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:21.269044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:21.317427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38bc2-1537-4a93-8a03-44987997c656 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:21.317734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38bc2-1537-4a93-8a03-44987997c656 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:21.318020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38bc2-1537-4a93-8a03-44987997c656 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:21.319648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 59/110] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:21.324810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23f1f7-69c6-4f74-9116-8c2606728083 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:21.325096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23f1f7-69c6-4f74-9116-8c2606728083 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:21.325420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23f1f7-69c6-4f74-9116-8c2606728083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:21.360263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:21.361631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5f23f1f7-69c6-4f74-9116-8c2606728083 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:21.363031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5f23f1f7-69c6-4f74-9116-8c2606728083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:21.400010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cf075-903b-48a9-873b-29a7ca0fbed1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:21.478579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:21.479399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:21.479959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:21.481473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:21.483500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:21.485748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:21.489387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 52/111] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:21 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:21.491086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:21.498121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 60/112] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:21.500902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23f1f7-69c6-4f74-9116-8c2606728083 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:21.610563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c613-7860-4d8e-a7e2-94b4a60ccdca None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:21.641352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:21.652343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:21.653068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:21.653504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:21.654822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:21.670211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:21.676946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:21.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:21.686032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:21.686996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:21.692092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23f1f7-69c6-4f74-9116-8c2606728083 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:21.694847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 53/113] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:21.701222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:21.757608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:21.758464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:21.758844 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:21.759222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:21.760071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] RBAC: Adding query filter params (role.id=24a475c8933346c6a1fa3359fbe0c818, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=eb70654945034353a5d54f660a61b97c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:21.760486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:21.762243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:21.769615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 61/114] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:21 2022] GET /identity/v3/role_assignments?user.id=eb70654945034353a5d54f660a61b97c&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=24a475c8933346c6a1fa3359fbe0c818 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:21.855411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:21.904818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881e68e-6ea1-4e48-b576-3f550b217549 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:21.976922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62c6f9-04ca-457f-830e-8b1e3b70a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:21.977945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62c6f9-04ca-457f-830e-8b1e3b70a781 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:21.978944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62c6f9-04ca-457f-830e-8b1e3b70a781 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:21.979442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62c6f9-04ca-457f-830e-8b1e3b70a781 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:21.980666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62c6f9-04ca-457f-830e-8b1e3b70a781 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:21.981207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62c6f9-04ca-457f-830e-8b1e3b70a781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:21.984553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62c6f9-04ca-457f-830e-8b1e3b70a781 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:22.003912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 54/115] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:22.005713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa083a-536c-466d-8264-5a5916068713 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.006039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa083a-536c-466d-8264-5a5916068713 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.006317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa083a-536c-466d-8264-5a5916068713 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.008032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 55/116] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:22.014989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa083a-536c-466d-8264-5a5916068713 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:22.074989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368c3ce-bcdf-4a58-8732-005df92fc6cb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:22.123955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:22.134512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b29643-3b0a-4e35-8750-839b892ee426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:22.135182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b29643-3b0a-4e35-8750-839b892ee426 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.135471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b29643-3b0a-4e35-8750-839b892ee426 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.135745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b29643-3b0a-4e35-8750-839b892ee426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.161789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b29643-3b0a-4e35-8750-839b892ee426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-67b29643-3b0a-4e35-8750-839b892ee426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:22.170805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:22.171888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:22.172866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:22.177394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 62/117] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:22.179890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce63b94-610f-475c-8f2b-2728f72c2639 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.183538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce63b94-610f-475c-8f2b-2728f72c2639 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.184016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce63b94-610f-475c-8f2b-2728f72c2639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.198338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:22.212713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa083a-536c-466d-8264-5a5916068713 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:22.222861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fce63b94-610f-475c-8f2b-2728f72c2639 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:22.226231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fce63b94-610f-475c-8f2b-2728f72c2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:22.296914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:22.297755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.298227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.298547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.299984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:22.301465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:22.312546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 56/118] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:22 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:22.313642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:22.337113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 63/119] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:22.340127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce63b94-610f-475c-8f2b-2728f72c2639 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:22.570045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a8591-89cb-46b0-9eac-79773af032ae None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:22.585606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce63b94-610f-475c-8f2b-2728f72c2639 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:22.621227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:22.629560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0883e-060e-48dc-a8e3-05aeeed3ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:22.630388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0883e-060e-48dc-a8e3-05aeeed3ce0c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.630775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0883e-060e-48dc-a8e3-05aeeed3ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.631157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0883e-060e-48dc-a8e3-05aeeed3ce0c None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.632472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0883e-060e-48dc-a8e3-05aeeed3ce0c None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:22.634309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0883e-060e-48dc-a8e3-05aeeed3ce0c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:22.641995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 64/120] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:22 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:22.642203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:22.644051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3205e-ddb1-4484-b5ec-02799b2b7f84 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.644447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3205e-ddb1-4484-b5ec-02799b2b7f84 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.644821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3205e-ddb1-4484-b5ec-02799b2b7f84 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.646489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 65/121] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:22.647187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3205e-ddb1-4484-b5ec-02799b2b7f84 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:22.659080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77121af0-997b-4f5e-bfa9-8148f9e243c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:22.659776 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121af0-997b-4f5e-bfa9-8148f9e243c7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.660064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121af0-997b-4f5e-bfa9-8148f9e243c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.660344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121af0-997b-4f5e-bfa9-8148f9e243c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.661022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77121af0-997b-4f5e-bfa9-8148f9e243c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:22.661328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77121af0-997b-4f5e-bfa9-8148f9e243c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:22.662750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77121af0-997b-4f5e-bfa9-8148f9e243c7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:22.684135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 57/122] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:22.686045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957740b-1fba-484b-bed8-edd2041fe862 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.686333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957740b-1fba-484b-bed8-edd2041fe862 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.686607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957740b-1fba-484b-bed8-edd2041fe862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.714403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c957740b-1fba-484b-bed8-edd2041fe862 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:22.715627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c957740b-1fba-484b-bed8-edd2041fe862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:22.795654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3205e-ddb1-4484-b5ec-02799b2b7f84 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:22.797509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 58/123] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:22.798403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957740b-1fba-484b-bed8-edd2041fe862 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:22.880101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:22.880742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:22.880971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:22.881249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/d20c9d80453a47e9990b3a60cdaa40ae/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:22.884368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb8255bb2f340b6aad60728a547a86c, user_id=d20c9d80453a47e9990b3a60cdaa40ae, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:22.885861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:22.910951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 66/124] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:22 2022] PUT /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/d20c9d80453a47e9990b3a60cdaa40ae/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:22.912885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:23.021035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957740b-1fba-484b-bed8-edd2041fe862 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:23.040301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:23.075228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:23.090557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:23.091643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:23.092072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:23.092437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f6cb8-1320-4824-ad7d-4df6ffeb4a3a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:23.092686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:23.109229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:23.115269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:23.116381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:23.117357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:23.119223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 59/125] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:23.126890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:23.140320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:23.141418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:23.142064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:23.142574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] PATH_INFO: `/v3/users/e4f47c952a6a40bf944493656ebcef14` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:23.146113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] RBAC: Authorizing `identity:get_user(user_id=e4f47c952a6a40bf944493656ebcef14)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:23.150118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:23.155013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 67/126] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:22 2022] GET /identity/v3/users/e4f47c952a6a40bf944493656ebcef14 => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:23.163314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:23.330713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d4157-cfd0-43bd-9921-06211e5d3cf7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:23.335717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc14b-30eb-422c-8fa9-dae400115239 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:23.389342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:23.401410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:23.402013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:23.402294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:23.402565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:23.404879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:23.405501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:23.405791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:23.406068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:23.406745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:23.407048 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:23.408713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:23.412901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:23.414618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:23.416693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 68/127] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:23 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:23.417243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:23.429046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 60/128] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:23.429901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:23.505069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:23.566958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa6759-ae8e-4d16-a6cb-31d4005adb0b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:23.604559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:23.605726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:23.606251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:23.607286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/e4f47c952a6a40bf944493656ebcef14/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:23.614759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb8255bb2f340b6aad60728a547a86c, user_id=e4f47c952a6a40bf944493656ebcef14, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:23.616490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:23.639598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 61/129] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:23 2022] PUT /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/e4f47c952a6a40bf944493656ebcef14/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:23.644496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:23.668522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e96f5f-ac7e-40c7-9f2d-c7707904df3e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:23.764724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:23.765863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:23.766360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:23.766848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:23.807570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:23.826963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:23.827962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:23.829080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:23.830291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 69/130] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:23.835926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:23.942264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff6d04-5705-4660-93ad-29f9b90a0a58 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:23.946125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:23.961173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:23.992165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321cfc4-c1ab-4fcd-abf0-715c8bead254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:23.992951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321cfc4-c1ab-4fcd-abf0-715c8bead254 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:23.993245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321cfc4-c1ab-4fcd-abf0-715c8bead254 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:23.993528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321cfc4-c1ab-4fcd-abf0-715c8bead254 None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:23.994869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321cfc4-c1ab-4fcd-abf0-715c8bead254 None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:23.996338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321cfc4-c1ab-4fcd-abf0-715c8bead254 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:23.998865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 62/131] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:23 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:24.000086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4106d3-1d87-4ff8-82df-429ade7dc7b7 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.000462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4106d3-1d87-4ff8-82df-429ade7dc7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.000747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4106d3-1d87-4ff8-82df-429ade7dc7b7 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.002133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 63/132] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:24.006082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4106d3-1d87-4ff8-82df-429ade7dc7b7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:24.027321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3b5e4-ad27-4964-8642-3697336521a1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:24.116190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc6e52-9aec-4929-9230-4822974a1ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:24.116828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc6e52-9aec-4929-9230-4822974a1ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.117114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc6e52-9aec-4929-9230-4822974a1ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.122063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc6e52-9aec-4929-9230-4822974a1ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.123445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc6e52-9aec-4929-9230-4822974a1ca8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:24.123769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc6e52-9aec-4929-9230-4822974a1ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:24.125233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc6e52-9aec-4929-9230-4822974a1ca8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:24.144610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 70/133] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:24.146082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e019f-9e31-4afc-b573-98ef41a7ab68 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.146375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e019f-9e31-4afc-b573-98ef41a7ab68 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.146658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e019f-9e31-4afc-b573-98ef41a7ab68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.176376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4106d3-1d87-4ff8-82df-429ade7dc7b7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:24.207069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6c8e019f-9e31-4afc-b573-98ef41a7ab68 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:24.210012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c8e019f-9e31-4afc-b573-98ef41a7ab68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:24.261119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:24.262108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.262394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.262676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] PATH_INFO: `/v3/projects/521a1e2f0e87407183cb9476b8214b58` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.264214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] RBAC: Authorizing `identity:get_project(project_id=521a1e2f0e87407183cb9476b8214b58)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:24.266853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:24.269212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 64/134] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:24 2022] GET /identity/v3/projects/521a1e2f0e87407183cb9476b8214b58 => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:24.270342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:24.369899 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 71/135] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:24.374891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:24.380706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e019f-9e31-4afc-b573-98ef41a7ab68 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:24.483345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:24.515887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097e1b3-679d-49b3-8085-a9fb7e7479ff None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:24.599885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e019f-9e31-4afc-b573-98ef41a7ab68 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:24.603342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:24.603342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.603694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.603906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.605218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:24.607286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:24.609003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 65/136] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:24 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:24.614828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:24.678523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78474555-a4d7-444f-9623-685e207f4935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:24.679218 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78474555-a4d7-444f-9623-685e207f4935 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.679497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78474555-a4d7-444f-9623-685e207f4935 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.679775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78474555-a4d7-444f-9623-685e207f4935 None admin] PATH_INFO: `/v3/projects/521a1e2f0e87407183cb9476b8214b58/users/d20c9d80453a47e9990b3a60cdaa40ae/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.687513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78474555-a4d7-444f-9623-685e207f4935 None admin] RBAC: Authorizing `identity:create_grant(project_id=521a1e2f0e87407183cb9476b8214b58, user_id=d20c9d80453a47e9990b3a60cdaa40ae, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:24.689020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78474555-a4d7-444f-9623-685e207f4935 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:24.720251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 72/137] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:24 2022] PUT /identity/v3/projects/521a1e2f0e87407183cb9476b8214b58/users/d20c9d80453a47e9990b3a60cdaa40ae/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:24.731242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a6013-9e94-4cff-bc05-3db65a283431 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.731435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a6013-9e94-4cff-bc05-3db65a283431 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.732851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a6013-9e94-4cff-bc05-3db65a283431 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.735684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 73/138] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:24.744455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d358-d302-408c-a096-e935e5bcb722 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.745467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d358-d302-408c-a096-e935e5bcb722 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.746061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d358-d302-408c-a096-e935e5bcb722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.783811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f819d358-d302-408c-a096-e935e5bcb722 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:24.785047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f819d358-d302-408c-a096-e935e5bcb722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:24.820857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3448cd6-7a1b-4a99-b6d9-210a57b8daec None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:24.846170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:24.928775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:24.929453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:24.929735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:24.930042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:24.932906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:24.934463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:24.938499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 66/139] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:24 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:24.939476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:24.960487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 74/140] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:24.961442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d358-d302-408c-a096-e935e5bcb722 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:25.031569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:25.136388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e6c7a-e7d2-40e6-a602-1f5bd75ec92b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:25.200223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d358-d302-408c-a096-e935e5bcb722 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:25.253604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:25.254315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:25.254585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:25.254869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:25.258485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:25.260082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:25.261963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 67/141] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:25.267645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:25.286629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:25.287686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:25.288178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:25.288659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/d20c9d80453a47e9990b3a60cdaa40ae/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:25.291321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb8255bb2f340b6aad60728a547a86c, user_id=d20c9d80453a47e9990b3a60cdaa40ae, role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:25.293069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:25.329238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 75/142] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:24 2022] PUT /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/d20c9d80453a47e9990b3a60cdaa40ae/roles/0a108a4463a4458480b00aed843b4e9b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:25.365851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:25.399907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:25.468674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9237137-51ef-4b6a-a421-4c2f242fcc8e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:25.546159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:25.547162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:25.552322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:25.552955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:25.553924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:25.555658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:25.593954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 68/143] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:43:25 2022] POST /identity/v3/users => generated 283 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:43:25.604201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:25.610754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47615106-c131-459d-b45b-0dc59425bc6f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:25.665598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:25.666436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:25.666727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:25.667009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:25.685500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:25.691015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:25.691994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:25.692793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 76/144] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:25.700295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:25.861839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:25.865711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e21cc-5046-4eb8-9a8c-d679475e0940 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:25.927626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fa734-fd6f-4b0b-bedc-66b033417034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:25.928910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fa734-fd6f-4b0b-bedc-66b033417034 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:25.929556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fa734-fd6f-4b0b-bedc-66b033417034 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:25.930257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fa734-fd6f-4b0b-bedc-66b033417034 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:25.931542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fa734-fd6f-4b0b-bedc-66b033417034 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:25.932240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fa734-fd6f-4b0b-bedc-66b033417034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:25.934349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fa734-fd6f-4b0b-bedc-66b033417034 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:25.951034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 69/145] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:25.957266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:26.088974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb068e9a-4925-4019-bee6-7c4acb4686eb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:26.181369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0bd2-d183-4a22-9f19-1735c81a3b6d None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:26.182208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0bd2-d183-4a22-9f19-1735c81a3b6d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:26.182547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0bd2-d183-4a22-9f19-1735c81a3b6d None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:26.184888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 70/146] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:26.185858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:26.186604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:26.186869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:26.187138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] PATH_INFO: `/v3/users/eb70654945034353a5d54f660a61b97c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:26.191806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd352-d043-4698-b430-021f14f72f05 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:26.192380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd352-d043-4698-b430-021f14f72f05 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:26.192872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd352-d043-4698-b430-021f14f72f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:26.194663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] RBAC: Authorizing `identity:get_user(user_id=eb70654945034353a5d54f660a61b97c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:26.196131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:26.198082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 77/147] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:25 2022] GET /identity/v3/users/eb70654945034353a5d54f660a61b97c => generated 289 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:26.205689 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:26.221003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-eaebd352-d043-4698-b430-021f14f72f05 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:26.225463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-eaebd352-d043-4698-b430-021f14f72f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:26.266288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:26.367218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 71/148] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:26.400378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ac05e-f2e6-463a-b6da-edd1787ed676 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:26.440643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:26.441202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:26.441477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:26.441747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:26.446471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:26.447997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:26.449878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 78/149] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:26 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:26.458229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebd352-d043-4698-b430-021f14f72f05 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:26.568277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:26.578535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebd352-d043-4698-b430-021f14f72f05 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:26.616936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:26.617609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:26.617917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:26.618189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/eb70654945034353a5d54f660a61b97c/roles/24a475c8933346c6a1fa3359fbe0c818` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:26.620162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb8255bb2f340b6aad60728a547a86c, user_id=eb70654945034353a5d54f660a61b97c, role_id=24a475c8933346c6a1fa3359fbe0c818)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:26.621581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:26.636892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 72/150] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:26 2022] PUT /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c/users/eb70654945034353a5d54f660a61b97c/roles/24a475c8933346c6a1fa3359fbe0c818 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:26.724309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:26.851291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:26.859562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acfd07-8710-4a06-b6e3-c35caabc8b1c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:26.901992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:26.902699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:26.902985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:26.903279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:26.916383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:26.921560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:26.922487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:26.923768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 79/151] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:26.928878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:27.045620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a075e5-1439-4fce-a3cf-c9e7a67d868a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:27.082103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf2590-7918-43ed-ae45-a5cf912e39f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:27.082787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf2590-7918-43ed-ae45-a5cf912e39f1 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:27.083068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf2590-7918-43ed-ae45-a5cf912e39f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:27.083391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf2590-7918-43ed-ae45-a5cf912e39f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:27.084126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf2590-7918-43ed-ae45-a5cf912e39f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:27.084435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf2590-7918-43ed-ae45-a5cf912e39f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:27.085828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf2590-7918-43ed-ae45-a5cf912e39f1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:27.095391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 73/152] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:27.101141 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:27.222311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a0d8-7089-4c35-91a3-639dc8296b96 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:27.222631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a0d8-7089-4c35-91a3-639dc8296b96 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:27.222867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a0d8-7089-4c35-91a3-639dc8296b96 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:27.224201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 74/153] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:27.230344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c108-4c5b-46e2-8f69-19d67f3e496c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:27.230626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c108-4c5b-46e2-8f69-19d67f3e496c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:27.230893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c108-4c5b-46e2-8f69-19d67f3e496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:27.245136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23efbb-61e3-4e95-aa17-fcd44079b364 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:27.258851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b091c108-4c5b-46e2-8f69-19d67f3e496c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:27.260078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b091c108-4c5b-46e2-8f69-19d67f3e496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:27.276232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:27.293449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:27.295053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:27.295505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:27.296267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:27.298003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:27.299674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:27.301484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 80/154] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:27 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:27.307892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:27.340982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:27.420692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 75/155] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:27.454308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d5d4d-1215-45e8-a392-8de45ccb0fca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:27.504569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:27.505192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:27.505649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:27.505952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:27.603555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:27.604948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:27.636652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:27.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 81/156] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:27.750763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c108-4c5b-46e2-8f69-19d67f3e496c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:27.868534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c108-4c5b-46e2-8f69-19d67f3e496c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:27.912357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:27.913002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:27.913285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:27.913559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:27.914753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:27.916186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:27.917852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 76/157] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:27 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:27.924048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:28.063979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b46db-2458-474f-84b0-61c9ba700795 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:28.092720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:28.104889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:28.105507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:28.105787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:28.106074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:28.107514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=d20c9d80453a47e9990b3a60cdaa40ae) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:28.107830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:28.109280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:28.118529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 82/158] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:27 2022] GET /identity/v3/role_assignments?user.id=d20c9d80453a47e9990b3a60cdaa40ae&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=3530f0bc598248418cc693afa8f4b150 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:28.139770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:28.156236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:28.257414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee7c0-1c5c-4ac3-a7f3-76837a4c009c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:28.297274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:28.298315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:28.298816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:28.299291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:28.311403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:28.317404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:28.318428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:28.319212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:28.321521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 77/159] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:28.331063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:28.496207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88027ae-ab9c-45e7-ae31-fb3dbd149bbe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:28.546186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:28.546812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:28.547091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:28.547365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:28.548057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:28.548360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:28.549784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:28.563802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 83/160] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:28.569450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:28.679663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:28.723458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf754122-de21-4cf2-95a5-ad1bed464485 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:28.772167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01328e6e-596c-4930-9865-50288294419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:28.772816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01328e6e-596c-4930-9865-50288294419e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:28.773096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01328e6e-596c-4930-9865-50288294419e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:28.773369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01328e6e-596c-4930-9865-50288294419e None admin] PATH_INFO: `/v3/users/e4f47c952a6a40bf944493656ebcef14` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:28.774524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01328e6e-596c-4930-9865-50288294419e None admin] RBAC: Authorizing `identity:get_user(user_id=e4f47c952a6a40bf944493656ebcef14)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:28.775971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01328e6e-596c-4930-9865-50288294419e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:28.781877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 78/161] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:28 2022] GET /identity/v3/users/e4f47c952a6a40bf944493656ebcef14 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:28.783994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:28.862742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:28.928660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d2bf-995c-42ff-92bd-5181723be4a9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:28.968087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:28.968700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:28.969179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:28.969454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:28.989674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267254a2-6e8c-405f-bd83-6afda776887b None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:28.990054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267254a2-6e8c-405f-bd83-6afda776887b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:28.990283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267254a2-6e8c-405f-bd83-6afda776887b None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:28.991676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 79/162] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:29.014134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dd1e6-c619-4a8b-8bdb-347f5e75a0fb None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:29.014442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dd1e6-c619-4a8b-8bdb-347f5e75a0fb None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:29.014680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dd1e6-c619-4a8b-8bdb-347f5e75a0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:29.043525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-316dd1e6-c619-4a8b-8bdb-347f5e75a0fb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:29.044672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-316dd1e6-c619-4a8b-8bdb-347f5e75a0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:29.086647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:29.088132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:29.180252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:29.224019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 80/163] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 11:43:29.251386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 84/164] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:29.257134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dd1e6-c619-4a8b-8bdb-347f5e75a0fb None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:29.369355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:29.406882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dd1e6-c619-4a8b-8bdb-347f5e75a0fb None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:29.459745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:29.460391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:29.460670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:29.460944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:29.462107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:29.463537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:29.465326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 81/165] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:29 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:29.473222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:29.604555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0692d-66e8-41ae-a41f-d2c512cc3f4a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:29.655150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:29.655686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:29.655966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:29.656236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:29.656971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=e4f47c952a6a40bf944493656ebcef14) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:29.657279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:29.659010 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:29.671236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 85/166] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:29 2022] GET /identity/v3/role_assignments?user.id=e4f47c952a6a40bf944493656ebcef14&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:29.836323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:29.958779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b61f-4f4c-41bd-9214-e1282c14ee74 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:30.006589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0dc5f-d1de-4946-bed4-d1f97485e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:30.007661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0dc5f-d1de-4946-bed4-d1f97485e7be None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.008152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0dc5f-d1de-4946-bed4-d1f97485e7be None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.008620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0dc5f-d1de-4946-bed4-d1f97485e7be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.016889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:30.026796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0dc5f-d1de-4946-bed4-d1f97485e7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f9c0dc5f-d1de-4946-bed4-d1f97485e7be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:30.032866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:30.034039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:30.034839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:30.037232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 82/167] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:30.043062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:30.060854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:30.069868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbcbb0-2c7d-43b4-88f2-6816555525ad None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.070149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbcbb0-2c7d-43b4-88f2-6816555525ad None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.071276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbcbb0-2c7d-43b4-88f2-6816555525ad None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.072853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 83/168] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:30.082892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27432c80-8603-4664-8192-e03478d98774 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.083398 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27432c80-8603-4664-8192-e03478d98774 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.083851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27432c80-8603-4664-8192-e03478d98774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.113586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-27432c80-8603-4664-8192-e03478d98774 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:30.115155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-27432c80-8603-4664-8192-e03478d98774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:30.250431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 84/169] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:30.260266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901dfe5-eac5-42cb-9131-286ed2f4c478 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:30.315922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:30.316696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.316987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.317267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.317983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:30.318297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:30.319682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:30.329377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 86/170] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:30.335106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27432c80-8603-4664-8192-e03478d98774 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:30.488492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27432c80-8603-4664-8192-e03478d98774 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:30.521840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:30.531635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:30.532275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.532611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.532827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.534005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:30.535446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:30.537338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 85/171] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:30 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:30.542622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:30.712215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:30.729992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:30.750329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32a402-16e4-492e-9096-0f9eb2f90b57 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:30.805275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd0ff4-dbac-48e9-8e52-0864650f5544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:30.805855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0ff4-dbac-48e9-8e52-0864650f5544 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.806142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0ff4-dbac-48e9-8e52-0864650f5544 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.806418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0ff4-dbac-48e9-8e52-0864650f5544 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.899698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a9c13-d86c-4b09-916f-73601c090ef5 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:30.948058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd0ff4-dbac-48e9-8e52-0864650f5544 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:30.948788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda77f0e-095f-47ee-8e84-fc756d3db707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:30.949499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda77f0e-095f-47ee-8e84-fc756d3db707 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.949814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda77f0e-095f-47ee-8e84-fc756d3db707 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.950240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda77f0e-095f-47ee-8e84-fc756d3db707 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.950614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd0ff4-dbac-48e9-8e52-0864650f5544 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:30.967256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda77f0e-095f-47ee-8e84-fc756d3db707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-eda77f0e-095f-47ee-8e84-fc756d3db707 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:30.975003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:30.976036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:30.976845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 86/172] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:30.978011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b049f-2002-40f3-82cb-8b5f7852cd8d None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.978447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b049f-2002-40f3-82cb-8b5f7852cd8d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.978979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b049f-2002-40f3-82cb-8b5f7852cd8d None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.980571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 87/173] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:30.988085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08cbe7-7fbc-4050-94ed-c517809b0961 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:30.989013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08cbe7-7fbc-4050-94ed-c517809b0961 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:30.989442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08cbe7-7fbc-4050-94ed-c517809b0961 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:30.991347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 88/174] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:30.992310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08cbe7-7fbc-4050-94ed-c517809b0961 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:31.043449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.059672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.073588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 87/175] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:31.075632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0577d8-d238-45e2-be94-cf09303f70b4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.075918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0577d8-d238-45e2-be94-cf09303f70b4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.076191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0577d8-d238-45e2-be94-cf09303f70b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.096763 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bf0577d8-d238-45e2-be94-cf09303f70b4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:31.097997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bf0577d8-d238-45e2-be94-cf09303f70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:31.111541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08cbe7-7fbc-4050-94ed-c517809b0961 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:31.149288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4843de-8321-4692-b163-beecc31fc4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:31.149939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4843de-8321-4692-b163-beecc31fc4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.150231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4843de-8321-4692-b163-beecc31fc4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.150502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4843de-8321-4692-b163-beecc31fc4c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.151182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4843de-8321-4692-b163-beecc31fc4c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:31.151490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4843de-8321-4692-b163-beecc31fc4c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:31.152847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4843de-8321-4692-b163-beecc31fc4c9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:31.161502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 89/176] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:31.162740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb5f05-1cb2-475b-874b-9a79f8cc1751 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.162998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb5f05-1cb2-475b-874b-9a79f8cc1751 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.163271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb5f05-1cb2-475b-874b-9a79f8cc1751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.164253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 88/177] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:31.165657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01181c32-005b-44f7-8546-6f6594d48d9a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.166094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01181c32-005b-44f7-8546-6f6594d48d9a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.166501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01181c32-005b-44f7-8546-6f6594d48d9a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.167740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 89/178] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:31.168419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01181c32-005b-44f7-8546-6f6594d48d9a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:31.192747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e6fb5f05-1cb2-475b-874b-9a79f8cc1751 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:31.194084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e6fb5f05-1cb2-475b-874b-9a79f8cc1751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:31.271671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 90/179] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:31.273002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b852424-2fac-4d0d-90da-e69444c7a787 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.273422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b852424-2fac-4d0d-90da-e69444c7a787 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.273570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b852424-2fac-4d0d-90da-e69444c7a787 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.275020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 91/180] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:31.275872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b852424-2fac-4d0d-90da-e69444c7a787 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:31.290272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01181c32-005b-44f7-8546-6f6594d48d9a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:31.293408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.333957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b4aea-4680-4d4b-8f3d-c01b7ed346b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:31.334907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4aea-4680-4d4b-8f3d-c01b7ed346b3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.335628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4aea-4680-4d4b-8f3d-c01b7ed346b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.336509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4aea-4680-4d4b-8f3d-c01b7ed346b3 None admin] PATH_INFO: `/v3/projects/521a1e2f0e87407183cb9476b8214b58` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.338135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b4aea-4680-4d4b-8f3d-c01b7ed346b3 None admin] RBAC: Authorizing `identity:get_project(project_id=521a1e2f0e87407183cb9476b8214b58)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:31.339051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.341660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b4aea-4680-4d4b-8f3d-c01b7ed346b3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:31.344755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 90/181] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:31 2022] GET /identity/v3/projects/521a1e2f0e87407183cb9476b8214b58 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:31.346058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee283f-3df5-4023-8783-4e1d24311dd1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.346469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee283f-3df5-4023-8783-4e1d24311dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.346866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee283f-3df5-4023-8783-4e1d24311dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.371717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-53ee283f-3df5-4023-8783-4e1d24311dd1 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:31.373758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-53ee283f-3df5-4023-8783-4e1d24311dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:31.414276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b852424-2fac-4d0d-90da-e69444c7a787 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:31.455577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 91/182] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:31.456997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323b935-f11a-4cc5-84cb-b6a6209d7864 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.457327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323b935-f11a-4cc5-84cb-b6a6209d7864 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.457602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323b935-f11a-4cc5-84cb-b6a6209d7864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.458736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:31.459675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.460161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.460867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] PATH_INFO: `/v3/users/d20c9d80453a47e9990b3a60cdaa40ae` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.462393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] RBAC: Authorizing `identity:get_user(user_id=d20c9d80453a47e9990b3a60cdaa40ae)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:31.464599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:31.467316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 92/183] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:31 2022] GET /identity/v3/users/d20c9d80453a47e9990b3a60cdaa40ae => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:31.468264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:31.477127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d323b935-f11a-4cc5-84cb-b6a6209d7864 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:31.478547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d323b935-f11a-4cc5-84cb-b6a6209d7864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:31.562707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 92/184] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:31.563994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323b935-f11a-4cc5-84cb-b6a6209d7864 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:31.598709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67737ae-ec61-4c8f-a60d-d57e173b562a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:31.601849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.623668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.649806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:31.650745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.651218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.651677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.653165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=521a1e2f0e87407183cb9476b8214b58, user.id=d20c9d80453a47e9990b3a60cdaa40ae) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:31.653671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:31.655895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:31.664935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 93/185] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:31 2022] GET /identity/v3/role_assignments?user.id=d20c9d80453a47e9990b3a60cdaa40ae&scope.project.id=521a1e2f0e87407183cb9476b8214b58&role.id=3530f0bc598248418cc693afa8f4b150 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:31.666257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:31.704579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323b935-f11a-4cc5-84cb-b6a6209d7864 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:31.756055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:31.757066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.757538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.758039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.784233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:31.789302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:31.790356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:31.791189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 93/186] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:31.796717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:31.829550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb5555-bc12-4c83-ae95-0a1ca9a3184b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:31.870862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:31.871394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.871669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.872601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.910933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.923432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08085f0c-92ba-40fb-91dd-b8e1dd2cdf1b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:31.937121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:31.961174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:31.962953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:31.968210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:31.968851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:31.969129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:31.969403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:31.983250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:31.988178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:31.989367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:31.990579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 94/187] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:31.991459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.069286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 94/188] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:32.070410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.093341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ccf08-b21f-4388-a2f0-8b27c31b8da5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.130086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.130742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.131018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.131283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.132390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.133850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.135400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 95/189] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.136146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.151344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.158377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.171194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177bd81-3861-46fa-a3cf-94c3bdb337fe None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.206545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.207156 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.207428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.207697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.208352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:32.208651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.210034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.218948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 95/190] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.219754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.241792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b32e3-f4ee-47c4-9183-836ff0ece7af None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.291263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.291944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.292165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.292438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.304447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:32.309557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:32.310570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:32.311381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 96/191] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:32.312068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.369350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ca6cd-020e-49c2-b98e-c1b5e65938c1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.396985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.403612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.404227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.404503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.404772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.405435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:32.405737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.407137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.415288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 96/192] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.416221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.417142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432247cd-f3e5-45ed-af21-f5653c72a8b8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.420703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.453901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.454528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.454817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.455133 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.456255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.457705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.459358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 97/193] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.461275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.519063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665f51a-c86c-4db5-8fd0-14db584feda9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.554300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.554908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.555193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.555463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.556134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.557571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.559934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e50677-0150-4aa3-af9c-e7d87799ad98 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.579710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 97/194] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:43:32 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:32.579710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.611064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.612037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.612524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.613000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] PATH_INFO: `/v3/users/eb70654945034353a5d54f660a61b97c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.615181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=eb70654945034353a5d54f660a61b97c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.617976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.625962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 98/195] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/users/eb70654945034353a5d54f660a61b97c => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.627173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.651609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.665438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.719444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca4ae0-7e18-4293-81fd-483c370d3956 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.755040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8d02-e1ae-40e3-98a5-70055d107ff7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.758056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.758680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.758961 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.759233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.759911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:32.760218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.761755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.771727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 98/196] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.772646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.804177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6636540-2118-41f8-a225-98be030babac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.805219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6636540-2118-41f8-a225-98be030babac None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.806663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6636540-2118-41f8-a225-98be030babac None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.806952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6636540-2118-41f8-a225-98be030babac None admin] PATH_INFO: `/v3/users/c547a14938c54e109295fb348db53c42` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.822368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6636540-2118-41f8-a225-98be030babac None admin] RBAC: Authorizing `identity:get_user(user_id=c547a14938c54e109295fb348db53c42)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.823845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6636540-2118-41f8-a225-98be030babac None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.826100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 99/197] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/users/c547a14938c54e109295fb348db53c42 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.827321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6636540-2118-41f8-a225-98be030babac None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:32.900947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a4ec3-940e-4471-b38e-2b8ea145bc17 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.921460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.937275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:32.959771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:32.960933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:32.961486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:32.961976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:32.963222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] RBAC: Adding query filter params (role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=d20c9d80453a47e9990b3a60cdaa40ae) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:32.963509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:32.965038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:32.974620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6636540-2118-41f8-a225-98be030babac None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:32.974875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 99/198] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/role_assignments?user.id=d20c9d80453a47e9990b3a60cdaa40ae&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=0a108a4463a4458480b00aed843b4e9b => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:32.976627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:33.031998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:33.034607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:33.034979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:33.035254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:33.128456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdceac-03b5-43b1-a5e6-24d2dc32f84e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:33.154471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:33.155954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:33.205826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:33.206030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:33.206254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:33.206428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] PATH_INFO: `/v3/users/c547a14938c54e109295fb348db53c42` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:33.208640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] RBAC: Authorizing `identity:get_user(user_id=c547a14938c54e109295fb348db53c42)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:33.211079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:33.219949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 100/199] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/users/c547a14938c54e109295fb348db53c42 => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:33.221786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:33.297856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:33.301626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:33.341016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 100/200] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:33.342686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:33.374833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5adad-2ce8-4cf5-8f79-74ddfe6ff978 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:33.414908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:33.415527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:33.415807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:33.416081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:33.473264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f82c72-d637-4e78-b799-4d67d66e6e89 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:33.503755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:33.505511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:33.521912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:33.522540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:33.524035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:33.524337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:33.587320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:33.592615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:33.626951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:33.629162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:33.651933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 101/201] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:33.652974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:33.774996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 101/202] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:33.776498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:33.788455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fad19f-9a92-44f9-990f-2024e14a0e55 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:33.833897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:33.834873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:33.835413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:33.835702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:33.836880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:33.839012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:33.841229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 102/203] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:33 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:33.842694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:33.880653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:33.885623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:33.924430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d7a5f-7879-42ba-b7cc-74f936ffb546 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:33.972809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:33.973718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:33.974041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:33.974320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:33.976124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:33.977588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:33.978206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b4459-13a4-4a67-804e-bfd1d13c1b3c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:33.979777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 102/204] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:33 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:33.980910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:34.016253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:34.016888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:34.017168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:34.017429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:34.018617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:34.020079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:34.026954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 103/205] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:33 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:34.026954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:34.129271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524d61-89a6-40e4-b7a4-993887b1d91a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:34.138746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:34.153943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941be3-29fe-4035-a579-a405373e00e8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:34.186706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:34.187892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:34.188359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:34.189039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:34.190536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] RBAC: Adding query filter params (role.id=24a475c8933346c6a1fa3359fbe0c818, scope.project.id=eeb8255bb2f340b6aad60728a547a86c, user.id=eb70654945034353a5d54f660a61b97c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:34.191099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:34.193029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:34.194697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c0cea-41ef-4a6a-abbe-e467fe0bb41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:34.195417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0cea-41ef-4a6a-abbe-e467fe0bb41f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:34.195797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0cea-41ef-4a6a-abbe-e467fe0bb41f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:34.196168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0cea-41ef-4a6a-abbe-e467fe0bb41f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:34.197308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c0cea-41ef-4a6a-abbe-e467fe0bb41f None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=c547a14938c54e109295fb348db53c42) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:34.197308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c0cea-41ef-4a6a-abbe-e467fe0bb41f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:34.198918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c0cea-41ef-4a6a-abbe-e467fe0bb41f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:34.207726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 103/206] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:33 2022] GET /identity/v3/role_assignments?user.id=eb70654945034353a5d54f660a61b97c&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=24a475c8933346c6a1fa3359fbe0c818 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:34.207726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:34.212912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 104/207] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:34 2022] GET /identity/v3/role_assignments?user.id=c547a14938c54e109295fb348db53c42&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:34.226203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:34.350230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e1ee5-7d60-4ffd-b914-dc582b9ffbc1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:34.409333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40040b52-3742-4a85-9ab4-6fae494fa5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:34.410255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040b52-3742-4a85-9ab4-6fae494fa5ad None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:34.410536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040b52-3742-4a85-9ab4-6fae494fa5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:34.410809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040b52-3742-4a85-9ab4-6fae494fa5ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:34.411641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040b52-3742-4a85-9ab4-6fae494fa5ad None admin] RBAC: Adding query filter params (role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=c547a14938c54e109295fb348db53c42) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:34.411955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040b52-3742-4a85-9ab4-6fae494fa5ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:34.413497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040b52-3742-4a85-9ab4-6fae494fa5ad None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:34.425085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 104/208] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:34 2022] GET /identity/v3/role_assignments?user.id=c547a14938c54e109295fb348db53c42&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=0a108a4463a4458480b00aed843b4e9b => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:35.097049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901eb9-9fa9-4b2f-82c4-b87caefbea5a None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:35.097583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901eb9-9fa9-4b2f-82c4-b87caefbea5a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:35.097583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901eb9-9fa9-4b2f-82c4-b87caefbea5a None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:35.099173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 105/209] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:35.104655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005bfa4-2880-4140-bc81-04bc5e0808b5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:35.104943 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005bfa4-2880-4140-bc81-04bc5e0808b5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:35.105218 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005bfa4-2880-4140-bc81-04bc5e0808b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:35.124475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4005bfa4-2880-4140-bc81-04bc5e0808b5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:35.125677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4005bfa4-2880-4140-bc81-04bc5e0808b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:35.206417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:35.212326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 105/210] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:35.261408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25577582-627d-47f9-a21d-5d9dd2dc63a9 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:35.261866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25577582-627d-47f9-a21d-5d9dd2dc63a9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:35.262147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25577582-627d-47f9-a21d-5d9dd2dc63a9 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:35.263991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 106/211] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:35.271093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11166-8519-4941-a679-3b14c18508b2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:35.271700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11166-8519-4941-a679-3b14c18508b2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:35.272179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11166-8519-4941-a679-3b14c18508b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:35.292721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f9e11166-8519-4941-a679-3b14c18508b2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:35.294907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f9e11166-8519-4941-a679-3b14c18508b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:35.372969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 106/212] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:35.492750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25577582-627d-47f9-a21d-5d9dd2dc63a9 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:35.616930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25577582-627d-47f9-a21d-5d9dd2dc63a9 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:35.644948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11166-8519-4941-a679-3b14c18508b2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:35.658771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:35.659459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:35.659852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:35.660151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:35.672087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:35.676955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:35.678159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:35.679423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 107/213] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:35.683925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:35.703735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:35.789803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11166-8519-4941-a679-3b14c18508b2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:35.799983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212ee86-3ec1-4b38-8257-c5374f41ae9b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:35.825524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:35.834178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:35.835163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:35.835920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:35.836365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:35.837866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:35.839708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:35.840809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:35.841493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:35.841872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:35.842234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:35.842525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 107/214] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:35.843030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:35.843438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:35.844916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:35.852731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:35.853920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 108/215] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:35.862063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:35.951895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:35.979297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f7897-6bd0-4f4f-90ac-ac5a24620dcb None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:35.982642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f8eb5-fe65-4fae-b7e4-07dfa9eace43 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:36.016589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:36.017223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.017498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.017843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] PATH_INFO: `/v3/users/f52679a4b943423d99d3443c5c3ba601` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.023528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:36.024196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.024476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.024733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.025386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:36.026851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:36.035764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] RBAC: Authorizing `identity:get_user(user_id=f52679a4b943423d99d3443c5c3ba601)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:36.037770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:36.044734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 109/216] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:35 2022] GET /identity/v3/users/f52679a4b943423d99d3443c5c3ba601 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:36.046093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:36.062804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 108/217] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:43:35 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:36.177675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15b63-6dc2-423d-a0b4-d9250dbb7e9e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:36.218343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48585a2-3d1f-469e-9253-246b17149df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:36.218883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48585a2-3d1f-469e-9253-246b17149df4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.219158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48585a2-3d1f-469e-9253-246b17149df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.219423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48585a2-3d1f-469e-9253-246b17149df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.222369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:36.296415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48585a2-3d1f-469e-9253-246b17149df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:36.297900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48585a2-3d1f-469e-9253-246b17149df4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:36.431056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 110/218] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:36.436835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:36.519614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:36.567727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063bd23-dca4-4bac-9ab7-722656eea059 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.567914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c90cd5-9165-4773-b8b5-5d0895ca5ab7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:36.567994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063bd23-dca4-4bac-9ab7-722656eea059 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.568280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063bd23-dca4-4bac-9ab7-722656eea059 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.573638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 111/219] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:36.575419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dcf43-3197-4d5d-bb55-029832340d10 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.575713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dcf43-3197-4d5d-bb55-029832340d10 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.575991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dcf43-3197-4d5d-bb55-029832340d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.594760 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-bd6dcf43-3197-4d5d-bb55-029832340d10 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:36.595817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-bd6dcf43-3197-4d5d-bb55-029832340d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:36.608645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:36.609789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.610267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.611030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.612806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:36.615418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:36.617878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 109/220] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:36 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:36.624409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:36.633789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:36.721352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 112/221] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:36.781245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3327-c43a-4311-a8d8-a6f44910b930 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:36.805206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:36.817645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961a3c5-ebdc-49d6-b295-0cfa8a9d4b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:36.818505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961a3c5-ebdc-49d6-b295-0cfa8a9d4b63 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.819056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961a3c5-ebdc-49d6-b295-0cfa8a9d4b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.819476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961a3c5-ebdc-49d6-b295-0cfa8a9d4b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.820468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961a3c5-ebdc-49d6-b295-0cfa8a9d4b63 None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=f52679a4b943423d99d3443c5c3ba601) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:36.821065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961a3c5-ebdc-49d6-b295-0cfa8a9d4b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:36.822670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961a3c5-ebdc-49d6-b295-0cfa8a9d4b63 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:36.829163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 110/222] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:36 2022] GET /identity/v3/role_assignments?user.id=f52679a4b943423d99d3443c5c3ba601&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=3530f0bc598248418cc693afa8f4b150 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:36.862878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e3603-eb3b-45b5-8769-c4bf394a7bd5 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.863159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e3603-eb3b-45b5-8769-c4bf394a7bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.863437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e3603-eb3b-45b5-8769-c4bf394a7bd5 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.864956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 113/223] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:36.870489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ef06-3a33-441e-991c-1ca0fb9bb2a2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:36.870998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ef06-3a33-441e-991c-1ca0fb9bb2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:36.871418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ef06-3a33-441e-991c-1ca0fb9bb2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:36.890457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f991ef06-3a33-441e-991c-1ca0fb9bb2a2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:36.892017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f991ef06-3a33-441e-991c-1ca0fb9bb2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:36.968319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 111/224] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:37.063196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e3603-eb3b-45b5-8769-c4bf394a7bd5 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:37.201011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e3603-eb3b-45b5-8769-c4bf394a7bd5 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:37.250805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:37.251451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:37.251734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:37.252012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:37.267468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ef06-3a33-441e-991c-1ca0fb9bb2a2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:37.271447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:37.276059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:37.276947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:37.277801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 114/225] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:37.285364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:37.327747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:37.389364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ef06-3a33-441e-991c-1ca0fb9bb2a2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:37.407078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:37.422800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4fb4c-65c6-499e-8265-e15022ae8ca7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:37.435578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:37.436233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:37.436514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:37.436787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:37.448201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:37.453152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:37.455580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:37.457268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 112/226] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:37.462713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:37.470928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:37.471565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:37.471849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:37.472125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:37.472831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:37.473136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:37.474628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:37.493833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 115/227] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:37.495090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:37.593812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:37.612828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a952904-0288-411b-8f09-4cad2cae2a8b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:37.650799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:37.651440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:37.651717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:37.651985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] PATH_INFO: `/v3/users/c547a14938c54e109295fb348db53c42` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:37.655297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] RBAC: Authorizing `identity:get_user(user_id=c547a14938c54e109295fb348db53c42)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:37.656766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:37.659517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 116/228] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:37 2022] GET /identity/v3/users/c547a14938c54e109295fb348db53c42 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:37.668092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:37.714026 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a33b7-f76c-4772-a1af-61ff9be67667 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:37.775721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:37.776721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:37.777201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:37.777664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:37.779123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:37.779623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:37.781999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:37.802599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 113/229] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:37.807685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:37.889802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:37.900973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78821f-cb19-4adb-b4d8-de7d21246e68 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:37.968044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:37.970345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:37.971079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:37.971899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:37.974169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:37.976578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:37.979241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 117/230] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:37 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:37.985714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff062885-b9b7-4196-822f-a004b6f2e9d3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:37.990597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:38.033423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:38.087054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:38.087710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:38.087995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:38.088273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] PATH_INFO: `/v3/users/c547a14938c54e109295fb348db53c42` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:38.094466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] RBAC: Authorizing `identity:get_user(user_id=c547a14938c54e109295fb348db53c42)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:38.095926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:38.097869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 114/231] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:37 2022] GET /identity/v3/users/c547a14938c54e109295fb348db53c42 => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:38.102895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:38.116421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ceb57a-934e-4087-902a-f8aa4593ac76 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:38.152626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:38.153559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:38.154071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:38.154544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1/users/c547a14938c54e109295fb348db53c42/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:38.156982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=67493e851cd54135a5319695e18189a1, user_id=c547a14938c54e109295fb348db53c42, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:38.158707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:38.176854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 118/232] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:37 2022] PUT /identity/v3/projects/67493e851cd54135a5319695e18189a1/users/c547a14938c54e109295fb348db53c42/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:38.284077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234207d6-21a2-4ed2-8c23-eee01ed3f37d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:38.292308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:38.330025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28122bc-37f5-47eb-8143-7a5315aa855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:38.331045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28122bc-37f5-47eb-8143-7a5315aa855e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:38.331527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28122bc-37f5-47eb-8143-7a5315aa855e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:38.332040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28122bc-37f5-47eb-8143-7a5315aa855e None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:38.333509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28122bc-37f5-47eb-8143-7a5315aa855e None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:38.335305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28122bc-37f5-47eb-8143-7a5315aa855e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:38.337498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 115/233] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:38 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:38.343653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:38.449581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:38.470003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302a1a-958e-4e20-b37e-234cad74c9f0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:38.507846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3f721-1e4f-4559-a749-04998efd09eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:38.508867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3f721-1e4f-4559-a749-04998efd09eb None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:38.509364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3f721-1e4f-4559-a749-04998efd09eb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:38.509861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3f721-1e4f-4559-a749-04998efd09eb None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1/users/c547a14938c54e109295fb348db53c42/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:38.512272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3f721-1e4f-4559-a749-04998efd09eb None admin] RBAC: Authorizing `identity:create_grant(project_id=67493e851cd54135a5319695e18189a1, user_id=c547a14938c54e109295fb348db53c42, role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:38.514063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3f721-1e4f-4559-a749-04998efd09eb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:38.531997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 119/234] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:38 2022] PUT /identity/v3/projects/67493e851cd54135a5319695e18189a1/users/c547a14938c54e109295fb348db53c42/roles/0a108a4463a4458480b00aed843b4e9b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:39.069305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b754136-ad61-4231-84d6-94ad41144144 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.069833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b754136-ad61-4231-84d6-94ad41144144 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.069912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b754136-ad61-4231-84d6-94ad41144144 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.071281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 116/235] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:39.077259 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4af074-3730-41be-8835-413381b9be8c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.077552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4af074-3730-41be-8835-413381b9be8c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.077865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4af074-3730-41be-8835-413381b9be8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.080617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c420e5-34a3-4d59-9aef-ddf59fa49ff7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.080904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c420e5-34a3-4d59-9aef-ddf59fa49ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.081177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c420e5-34a3-4d59-9aef-ddf59fa49ff7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.082785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 117/236] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:39.095823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856602e-0033-4de2-84b6-575e64223681 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.096251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856602e-0033-4de2-84b6-575e64223681 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.096668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856602e-0033-4de2-84b6-575e64223681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.104087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-1f4af074-3730-41be-8835-413381b9be8c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:39.104087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1f4af074-3730-41be-8835-413381b9be8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:39.135697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7856602e-0033-4de2-84b6-575e64223681 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:39.136907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7856602e-0033-4de2-84b6-575e64223681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:39.228607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 120/237] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:39.265361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 118/238] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:39.439126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416a91-9050-4356-adb3-c4a4b595880a None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.439609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416a91-9050-4356-adb3-c4a4b595880a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.439609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416a91-9050-4356-adb3-c4a4b595880a None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.440944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 121/239] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:39.447123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf168-dd39-447e-af12-eff5a0145ca7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.447412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf168-dd39-447e-af12-eff5a0145ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.447693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf168-dd39-447e-af12-eff5a0145ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.477529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-22dbf168-dd39-447e-af12-eff5a0145ca7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:39.478761 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22dbf168-dd39-447e-af12-eff5a0145ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:39.564359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 119/240] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:39.581426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416a91-9050-4356-adb3-c4a4b595880a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:39.623142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf168-dd39-447e-af12-eff5a0145ca7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:39.682976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:39.717571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:39.776470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416a91-9050-4356-adb3-c4a4b595880a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:39.788894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf168-dd39-447e-af12-eff5a0145ca7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:39.843175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:39.843815 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.844096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.844369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.845967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:39.846652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:39.846941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:39.847205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:39.848473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:39.849944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:39.856171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 122/241] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:39.858926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:39.869339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:39.874741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:39.875662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:39.876576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:39.880767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 120/242] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:39.882648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:39.998822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64264f87-6f78-4f7d-89f0-2356a8aa0ed8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.003013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:40.011326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522819ba-6e75-4016-b22e-5907a86aecbc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.043973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.044592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.044876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.045149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.045824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.047446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.052431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.053324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.053867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.054339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.055210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:40.055715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.057455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.059454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:40.071144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 121/243] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:40.072660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.075889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 123/244] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:43:39 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:40.078952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.208107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093d160-17a4-4d7d-8756-0c404f65f6a1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.239352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1bd45-6068-498b-83e8-5f54f6f32783 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.246140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.246925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.247217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.247491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] PATH_INFO: `/v3/users/7d29e8b91fdf4d50be02c31aab2c20c9` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.266556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] RBAC: Authorizing `identity:get_user(user_id=7d29e8b91fdf4d50be02c31aab2c20c9)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.268947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.271073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 124/245] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/users/7d29e8b91fdf4d50be02c31aab2c20c9 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:40.277041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.309885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.310522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.310800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.311069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.323071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:40.329129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:40.330088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:40.330947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 122/246] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:40.342639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.346142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:40.390092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:40.408872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ef3f2-870c-4be9-897f-efdc18cf3a5d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.449442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.451471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.452000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.452525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.504385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbf31e-3ecd-444d-be28-f4420aa16482 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.537982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.540179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.545623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.546291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.546555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.546829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.547502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:40.547806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.549220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.565405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 123/247] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:40.566588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.611410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:40.662924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:40.673379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 125/248] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:43:40.680242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.699964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b16a6f-3278-469f-adad-1cc83fe4306d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.742064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.742785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.743137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.743479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] PATH_INFO: `/v3/users/f52679a4b943423d99d3443c5c3ba601` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.745571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] RBAC: Authorizing `identity:get_user(user_id=f52679a4b943423d99d3443c5c3ba601)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.747888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.753786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 124/249] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/users/f52679a4b943423d99d3443c5c3ba601 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:40.755468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.815643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa75c2-6d6a-41d5-b621-2500a31838b0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.864600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.865677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.866172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.866633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.868935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.870584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.873569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 126/250] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:40.880099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.881718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838eb5e-759c-415f-8c45-11620654f4c5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:40.911283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:40.929956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:40.930507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:40.930794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:40.931071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:40.934365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:40.935595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:40.937619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 125/251] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:40.959574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:40.986485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:41.004737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16ff97-6577-4cd6-9c2e-dfa3abb30d9d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:41.052357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43c98d-3673-4f90-928e-059e799392b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:41.053352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c98d-3673-4f90-928e-059e799392b5 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:41.053852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c98d-3673-4f90-928e-059e799392b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:41.054316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c98d-3673-4f90-928e-059e799392b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:41.055267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43c98d-3673-4f90-928e-059e799392b5 None admin] RBAC: Adding query filter params (role.id=24a475c8933346c6a1fa3359fbe0c818, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=7d29e8b91fdf4d50be02c31aab2c20c9) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:41.056049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43c98d-3673-4f90-928e-059e799392b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:41.059123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43c98d-3673-4f90-928e-059e799392b5 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:41.072110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 127/252] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:40 2022] GET /identity/v3/role_assignments?user.id=7d29e8b91fdf4d50be02c31aab2c20c9&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=24a475c8933346c6a1fa3359fbe0c818 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:41.128785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233fca4-cc75-4fe0-89e1-697ea6d62b24 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:41.188682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e90f3c-a031-4cc7-918c-0b1af536118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:41.189355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e90f3c-a031-4cc7-918c-0b1af536118a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:41.189633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e90f3c-a031-4cc7-918c-0b1af536118a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:41.189930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e90f3c-a031-4cc7-918c-0b1af536118a None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1/users/f52679a4b943423d99d3443c5c3ba601/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:41.191889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e90f3c-a031-4cc7-918c-0b1af536118a None admin] RBAC: Authorizing `identity:create_grant(project_id=67493e851cd54135a5319695e18189a1, user_id=f52679a4b943423d99d3443c5c3ba601, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:41.193376 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e90f3c-a031-4cc7-918c-0b1af536118a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:41.209454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8c58-09a6-4c09-b1e2-335909e146db None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:41.210241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8c58-09a6-4c09-b1e2-335909e146db None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:41.210713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8c58-09a6-4c09-b1e2-335909e146db None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:41.214298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 128/253] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:41.219596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f98fa-aef5-4f4e-a165-b6ef59b65500 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:41.220030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 126/254] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:40 2022] PUT /identity/v3/projects/67493e851cd54135a5319695e18189a1/users/f52679a4b943423d99d3443c5c3ba601/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:43:41.220436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f98fa-aef5-4f4e-a165-b6ef59b65500 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:41.220930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f98fa-aef5-4f4e-a165-b6ef59b65500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:41.251085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d64f98fa-aef5-4f4e-a165-b6ef59b65500 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:41.252747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d64f98fa-aef5-4f4e-a165-b6ef59b65500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:41.367999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6011f1-9210-4292-9e5e-96fc4775dec3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:41.368295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6011f1-9210-4292-9e5e-96fc4775dec3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:41.368457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6011f1-9210-4292-9e5e-96fc4775dec3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:41.370103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 127/255] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:41.380416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 129/256] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:41.384084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3e5ca-7493-4c2c-a31a-633780963b8f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:41.384363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3e5ca-7493-4c2c-a31a-633780963b8f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:41.384634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3e5ca-7493-4c2c-a31a-633780963b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:41.422600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b2f3e5ca-7493-4c2c-a31a-633780963b8f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:41.423862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b2f3e5ca-7493-4c2c-a31a-633780963b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:41.527112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 128/257] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:41.802934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f98fa-aef5-4f4e-a165-b6ef59b65500 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:41.826897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:41.946269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f98fa-aef5-4f4e-a165-b6ef59b65500 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:41.993413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3e5ca-7493-4c2c-a31a-633780963b8f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:41.997197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:42.016878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:42.017977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:42.018434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:42.018885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:42.045708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:42.051116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:42.052009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:42.052845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:42.054845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 130/258] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:42.060863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:42.154111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3e5ca-7493-4c2c-a31a-633780963b8f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:42.238213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:42.239430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:42.240146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:42.240440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:42.240716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:42.262183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab464f0-02fd-4958-b8d2-0fe11f799810 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:42.271225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:42.276656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:42.277739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:42.278630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:42.286353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 129/259] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:42.292244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:42.334646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:42.335186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:42.335466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:42.335742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:42.336427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:42.336729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:42.338127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:42.352337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 131/260] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:42.358794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:42.418129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:42.490388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4db35-98c6-4b30-b280-1d3e18ab4803 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:42.504128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2dedd-a72a-47ba-b8f3-446cb16e3bb0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:42.570856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:42.571509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:42.571794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:42.572073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] PATH_INFO: `/v3/users/c547a14938c54e109295fb348db53c42` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:42.573391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] RBAC: Authorizing `identity:get_user(user_id=c547a14938c54e109295fb348db53c42)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:42.576120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:42.577919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 132/261] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:42 2022] GET /identity/v3/users/c547a14938c54e109295fb348db53c42 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:42.584313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:42.596941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:42.600566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:42.601193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:42.601471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:42.601744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:42.602446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:42.602749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:42.604619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:42.625189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 130/262] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:42.630981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:42.808920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241af622-b72d-456e-8548-dd99a3374e02 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:42.839881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3b198-0fb0-4ddf-9cfd-0ec72a4ad4a8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:42.885023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:42.885623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:42.885935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:42.886216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:42.911542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:42.950958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:42.952001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:42.952491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:42.952966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] PATH_INFO: `/v3/users/c547a14938c54e109295fb348db53c42` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:42.954648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] RBAC: Authorizing `identity:get_user(user_id=c547a14938c54e109295fb348db53c42)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:42.956284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:42.958139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 131/263] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:42 2022] GET /identity/v3/users/c547a14938c54e109295fb348db53c42 => generated 283 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:42.963595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:43.019180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:43.020806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:43.033131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:43.209870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 133/264] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:43.217659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:43.219919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e55df9-8044-4099-b69f-4e2a06202d56 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:43.299921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:43.300904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:43.301406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:43.301912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:43.410183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8081-a0d1-435b-a63b-6cc4dc353c1d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:43.426987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:43.447528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:43.461027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:43.462534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:43.498757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:43.499427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:43.499710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:43.499984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:43.501257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:43.503211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:43.504872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 134/265] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:43 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:43.510872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:43.629701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 132/266] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:43:43.635693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:43.744185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79278-0874-4fb3-9ae4-f8c425254e75 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:43.805313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:43.817813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d94b4b-fdec-4b03-b391-8d6ab0605f4f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:43.822328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8b8b7-a8a1-455c-9861-8231b63777b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:43.823486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8b8b7-a8a1-455c-9861-8231b63777b8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:43.823914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8b8b7-a8a1-455c-9861-8231b63777b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:43.824333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8b8b7-a8a1-455c-9861-8231b63777b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:43.825741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8b8b7-a8a1-455c-9861-8231b63777b8 None admin] RBAC: Adding query filter params (role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=c547a14938c54e109295fb348db53c42) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:43.826218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8b8b7-a8a1-455c-9861-8231b63777b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:43.827981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8b8b7-a8a1-455c-9861-8231b63777b8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:43.841361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 135/267] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:43 2022] GET /identity/v3/role_assignments?user.id=c547a14938c54e109295fb348db53c42&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=0a108a4463a4458480b00aed843b4e9b => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:43.843081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cddda-29cc-42a6-911c-ae2730034325 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:43.843692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cddda-29cc-42a6-911c-ae2730034325 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:43.844168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cddda-29cc-42a6-911c-ae2730034325 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:43.845687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 136/268] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:43.867340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6549273-a75b-4cf2-b10c-3a32ec4ff00a None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:43.869364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6549273-a75b-4cf2-b10c-3a32ec4ff00a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:43.869796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6549273-a75b-4cf2-b10c-3a32ec4ff00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:43.879956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:43.880637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:43.880924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:43.881201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:43.882736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:43.884293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:43.886553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 133/269] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:43 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:43.893073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:43.906740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c6549273-a75b-4cf2-b10c-3a32ec4ff00a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:43.908067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c6549273-a75b-4cf2-b10c-3a32ec4ff00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:44.015307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:44.069261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f34e1-909e-4391-8417-65d8207ce848 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:44.075377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 137/270] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:44.076840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440db8-f2a8-4181-9847-a980a4fe7cb5 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.081592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440db8-f2a8-4181-9847-a980a4fe7cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.081592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440db8-f2a8-4181-9847-a980a4fe7cb5 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.082262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 138/271] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:44.088243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2859-50c0-494e-9a99-83f3531b0578 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.089110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2859-50c0-494e-9a99-83f3531b0578 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.089569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2859-50c0-494e-9a99-83f3531b0578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.112223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b2db2859-50c0-494e-9a99-83f3531b0578 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:44.113875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b2db2859-50c0-494e-9a99-83f3531b0578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:44.145364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:44.146073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.146350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.146620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.147364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=c547a14938c54e109295fb348db53c42) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:44.147676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:44.149130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:44.166235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 134/272] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:43 2022] GET /identity/v3/role_assignments?user.id=c547a14938c54e109295fb348db53c42&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:44.216886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 139/273] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:44.400377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:44.432954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:44.478231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985529e1-c50d-4baa-bdd1-df94df8317db None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.478891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985529e1-c50d-4baa-bdd1-df94df8317db None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.480051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985529e1-c50d-4baa-bdd1-df94df8317db None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.484255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 140/274] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:44.490500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19662e-e080-4cdd-abbd-c7c4c636be55 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.490866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19662e-e080-4cdd-abbd-c7c4c636be55 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.491371 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19662e-e080-4cdd-abbd-c7c4c636be55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.513509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0b19662e-e080-4cdd-abbd-c7c4c636be55 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:44.514900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0b19662e-e080-4cdd-abbd-c7c4c636be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:44.536517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc89def-5c4a-4b70-9b9e-68ce27117172 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:44.579770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed8f75-cc07-4adf-b11f-9cf23e82e196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:44.580858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed8f75-cc07-4adf-b11f-9cf23e82e196 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.581410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed8f75-cc07-4adf-b11f-9cf23e82e196 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.581686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed8f75-cc07-4adf-b11f-9cf23e82e196 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.596261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed8f75-cc07-4adf-b11f-9cf23e82e196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:44.602789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 141/275] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7eed8f75-cc07-4adf-b11f-9cf23e82e196 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:44.603412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:44.604375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:44.605192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 135/276] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:44.612284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19662e-e080-4cdd-abbd-c7c4c636be55 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:44.618998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78344-c963-40d3-9d26-e6286b227876 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.619290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78344-c963-40d3-9d26-e6286b227876 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.619565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab78344-c963-40d3-9d26-e6286b227876 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.621150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 136/277] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:44.627184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ba8da-b4d9-4552-9c8f-9459277a5e29 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.627438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ba8da-b4d9-4552-9c8f-9459277a5e29 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.627856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ba8da-b4d9-4552-9c8f-9459277a5e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.649956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:44.654625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fe7ba8da-b4d9-4552-9c8f-9459277a5e29 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:44.655770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fe7ba8da-b4d9-4552-9c8f-9459277a5e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:44.729183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 137/278] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:44.729977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ba8da-b4d9-4552-9c8f-9459277a5e29 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:44.745116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19662e-e080-4cdd-abbd-c7c4c636be55 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:44.762698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:44.787472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:44.788609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.789455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.789908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.791008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:44.791462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:44.793204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:44.803991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 142/279] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:44.810176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:44.845300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ba8da-b4d9-4552-9c8f-9459277a5e29 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:44.890264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:44.891252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.891533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.891807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.892928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:44.894418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:44.896020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 138/280] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:44.901468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:44.914968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:44.927543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec19a7-1dba-413b-b63b-91da25d25b49 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:44.974461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:44.975093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:44.975374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:44.975651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] PATH_INFO: `/v3/users/f0bb01ed7d264537ab955dbb4b9c3219` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:44.989408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] RBAC: Authorizing `identity:get_user(user_id=f0bb01ed7d264537ab955dbb4b9c3219)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:44.990887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:44.992726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 143/281] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:44 2022] GET /identity/v3/users/f0bb01ed7d264537ab955dbb4b9c3219 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:44.995638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.007902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:45.024649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c05d8a-d9d5-4db8-95ee-97513d70d7a4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.058389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.059517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.059793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.060059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.070569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:45.075996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:45.077037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 139/282] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:45.082932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.104938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409dce3-a3b1-4ed8-a470-abf41df329af None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.140712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.141317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.141630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.141939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.142592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:45.144008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:45.165648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 144/283] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:43:44 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:45.170827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.178903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a196f-aaf0-4354-86f9-453c6013fb00 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.186299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:45.226223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.226954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.227235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.227767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.228250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, user.id=f0bb01ed7d264537ab955dbb4b9c3219, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:45.228560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:45.230801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:45.245826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 140/284] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/role_assignments?user.id=f0bb01ed7d264537ab955dbb4b9c3219&scope.system=all&role.id=3530f0bc598248418cc693afa8f4b150 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:45.245826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.252438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:45.320325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb2d93-3e72-46a0-b20b-0d8c9b0f6432 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.360263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5fa4e-5a0d-4658-a83a-5fe1b693c1ea None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.374591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.375336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.375743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.376019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.393016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:45.398022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:45.398984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:45.399847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:45.404448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 145/285] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:45.405453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.412879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.413541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.413773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.414063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.414754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:45.415055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:45.416543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:45.428142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 141/286] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:45.433424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.490976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:45.516968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:45.534423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9332ea1-3101-4920-86a2-98e36fe3162a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.555189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d8b8d-9d92-47b6-b83f-f78a154d7841 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.579248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.579993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.580278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.581034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.581723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:45.582057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:45.583422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:45.594954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 146/287] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:45.600213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.606784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.607634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.608062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.608475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] PATH_INFO: `/v3/users/7d29e8b91fdf4d50be02c31aab2c20c9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.609826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] RBAC: Authorizing `identity:get_user(user_id=7d29e8b91fdf4d50be02c31aab2c20c9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:45.612307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:45.615085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 142/288] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/users/7d29e8b91fdf4d50be02c31aab2c20c9 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:45.621408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.745494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c75adf-3a2b-41c5-a308-a2db7388d808 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.768684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7b905-af9a-4068-825e-7cf409ab4905 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:45.779605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:45.791055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.792034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.792489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.793046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] PATH_INFO: `/v3/users/f52679a4b943423d99d3443c5c3ba601` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.794460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] RBAC: Authorizing `identity:get_user(user_id=f52679a4b943423d99d3443c5c3ba601)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:45.796378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:45.798595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 147/289] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/users/f52679a4b943423d99d3443c5c3ba601 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:45.806790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.844940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:45.845658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:45.847613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:45.847902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:45.849811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:45.851308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:45.854331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 143/290] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:45.861386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:45.873605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:46.005905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc819-5ca1-42b6-87b4-4e19163546d8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:46.016180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d165-f070-4d86-b383-0c31bf13a9c7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:46.066318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:46.066923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:46.067199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:46.067455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:46.074533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:46.075740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:46.076016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:46.076289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1/users/7d29e8b91fdf4d50be02c31aab2c20c9/roles/24a475c8933346c6a1fa3359fbe0c818` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:46.078357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] RBAC: Authorizing `identity:create_grant(project_id=67493e851cd54135a5319695e18189a1, user_id=7d29e8b91fdf4d50be02c31aab2c20c9, role_id=24a475c8933346c6a1fa3359fbe0c818)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:46.080084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:46.101544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 144/291] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:43:45 2022] PUT /identity/v3/projects/67493e851cd54135a5319695e18189a1/users/7d29e8b91fdf4d50be02c31aab2c20c9/roles/24a475c8933346c6a1fa3359fbe0c818 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:46.146241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:46.160140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:46.161596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:46.308185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 148/292] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:46.314425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:46.412487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:46.457391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffd6bb-5431-4df4-9ee8-8d985cb8d678 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:46.494134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d8b6d-8060-4b9b-8969-60c9a61a9605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:46.495134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8b6d-8060-4b9b-8969-60c9a61a9605 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:46.495606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8b6d-8060-4b9b-8969-60c9a61a9605 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:46.496071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8b6d-8060-4b9b-8969-60c9a61a9605 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:46.497953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d8b6d-8060-4b9b-8969-60c9a61a9605 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:46.499704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d8b6d-8060-4b9b-8969-60c9a61a9605 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:46.501848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 145/293] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:46 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:46.507722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:46.617632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:46.628732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba59d50-697e-4c8b-a1a0-dad5828114f7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:46.673249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf0698-c817-47f8-b9ab-bcb0c1017109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:46.673878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf0698-c817-47f8-b9ab-bcb0c1017109 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:46.674155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf0698-c817-47f8-b9ab-bcb0c1017109 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:46.674418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf0698-c817-47f8-b9ab-bcb0c1017109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:46.675143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf0698-c817-47f8-b9ab-bcb0c1017109 None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=f52679a4b943423d99d3443c5c3ba601) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:46.675446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf0698-c817-47f8-b9ab-bcb0c1017109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:46.676870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf0698-c817-47f8-b9ab-bcb0c1017109 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:46.687446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 149/294] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:46 2022] GET /identity/v3/role_assignments?user.id=f52679a4b943423d99d3443c5c3ba601&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=3530f0bc598248418cc693afa8f4b150 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:47.734541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3281-a178-4676-a5d1-c64cd96a30c4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:47.735058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3281-a178-4676-a5d1-c64cd96a30c4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:47.735058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3281-a178-4676-a5d1-c64cd96a30c4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:47.736824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 146/295] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:47.742885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f294f-ce1b-4d36-aba9-310f805c55be None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:47.743327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f294f-ce1b-4d36-aba9-310f805c55be None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:47.743402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f294f-ce1b-4d36-aba9-310f805c55be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:47.762747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-7b6f294f-ce1b-4d36-aba9-310f805c55be None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:47.763928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7b6f294f-ce1b-4d36-aba9-310f805c55be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:47.785653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dbfbe-3cf1-442b-8c61-e5b7f4a589e6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:47.786015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dbfbe-3cf1-442b-8c61-e5b7f4a589e6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:47.786232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dbfbe-3cf1-442b-8c61-e5b7f4a589e6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:47.787693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 147/296] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:47.793874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e00b73-9426-4e6e-8ed6-be970103a3be None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:47.794448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e00b73-9426-4e6e-8ed6-be970103a3be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:47.794923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e00b73-9426-4e6e-8ed6-be970103a3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:47.814539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-88e00b73-9426-4e6e-8ed6-be970103a3be None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:47.816087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-88e00b73-9426-4e6e-8ed6-be970103a3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:47.850618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 150/297] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:47.898738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 148/298] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:48.139455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f294f-ce1b-4d36-aba9-310f805c55be None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.186470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e00b73-9426-4e6e-8ed6-be970103a3be None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.243000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:48.259343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f294f-ce1b-4d36-aba9-310f805c55be None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:48.259850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:48.298235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e00b73-9426-4e6e-8ed6-be970103a3be None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:48.298709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6e996-362e-4d25-8b60-85779789458f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:48.299363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e996-362e-4d25-8b60-85779789458f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.299752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e996-362e-4d25-8b60-85779789458f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.300020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e996-362e-4d25-8b60-85779789458f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.311322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6e996-362e-4d25-8b60-85779789458f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-eea6e996-362e-4d25-8b60-85779789458f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:48.316560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:48.317465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 151/299] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:48.323309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aaace-6040-4f39-b2cc-f67ecbfef909 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.324267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aaace-6040-4f39-b2cc-f67ecbfef909 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.324267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aaace-6040-4f39-b2cc-f67ecbfef909 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.324267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 152/300] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:48.327184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aaace-6040-4f39-b2cc-f67ecbfef909 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.339314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86511f-45e4-4eef-9088-fd6c6210f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:48.339950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86511f-45e4-4eef-9088-fd6c6210f4de None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.340304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86511f-45e4-4eef-9088-fd6c6210f4de None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.340587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86511f-45e4-4eef-9088-fd6c6210f4de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.352096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86511f-45e4-4eef-9088-fd6c6210f4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2d86511f-45e4-4eef-9088-fd6c6210f4de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:48.357602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:48.359190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:48.360840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:48.363803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 149/301] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:48.365214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc087a66-30a0-4efc-ba49-37a5f1299f1d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.365497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc087a66-30a0-4efc-ba49-37a5f1299f1d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.365809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc087a66-30a0-4efc-ba49-37a5f1299f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.386898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dc087a66-30a0-4efc-ba49-37a5f1299f1d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:48.388155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc087a66-30a0-4efc-ba49-37a5f1299f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:48.467831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aaace-6040-4f39-b2cc-f67ecbfef909 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:48.479237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 150/302] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:48.479237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc087a66-30a0-4efc-ba49-37a5f1299f1d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.516110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:48.516723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.517005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.517385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.518189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:48.518503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:48.519870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:48.533632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 153/303] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:48.535041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.545132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:48.551930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:48.612059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc087a66-30a0-4efc-ba49-37a5f1299f1d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:48.658051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be5302-f3a8-4774-a75c-e7f508c536e1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:48.659469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:48.660495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.660772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.662096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.662838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:48.663152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:48.664577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:48.675803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 151/304] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:48.683124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.706059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:48.707445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.707942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.708385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] PATH_INFO: `/v3/users/f0bb01ed7d264537ab955dbb4b9c3219` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.709862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] RBAC: Authorizing `identity:get_user(user_id=f0bb01ed7d264537ab955dbb4b9c3219)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:48.712433 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:48.714766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 154/305] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/users/f0bb01ed7d264537ab955dbb4b9c3219 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:48.722680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.814458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0028c9-59a3-4b67-bb60-0204d82aea02 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:48.818230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:48.832407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:48.841185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59ed43-dffd-4e5a-ba9e-6e92d87b347a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:48.849988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:48.850625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.850905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.851179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] PATH_INFO: `/v3/users/9434caec0e9e48f280f16bacbd50d865` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.864317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] RBAC: Authorizing `identity:get_user(user_id=9434caec0e9e48f280f16bacbd50d865)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:48.865775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:48.872673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 152/306] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/users/9434caec0e9e48f280f16bacbd50d865 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:48.872673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:48.876284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfc4b3-08d8-4e37-bff3-1bfa2e76d76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:48.877014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc4b3-08d8-4e37-bff3-1bfa2e76d76c None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.877363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc4b3-08d8-4e37-bff3-1bfa2e76d76c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.877648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc4b3-08d8-4e37-bff3-1bfa2e76d76c None admin] PATH_INFO: `/v3/system/users/f0bb01ed7d264537ab955dbb4b9c3219/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.879096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfc4b3-08d8-4e37-bff3-1bfa2e76d76c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f0bb01ed7d264537ab955dbb4b9c3219, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:48.880470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfc4b3-08d8-4e37-bff3-1bfa2e76d76c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:48.891391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 155/307] 10.222.0.88 () {58 vars in 1334 bytes} [Mon Sep 19 11:43:48 2022] PUT /identity/v3/system/users/f0bb01ed7d264537ab955dbb4b9c3219/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:43:48.892776 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34db3ea-c2ba-4b5a-933f-ee78aac58d5e None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:48.893068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34db3ea-c2ba-4b5a-933f-ee78aac58d5e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:48.893347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34db3ea-c2ba-4b5a-933f-ee78aac58d5e None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:48.894646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 156/308] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:48.895378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34db3ea-c2ba-4b5a-933f-ee78aac58d5e None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:49.012906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18965b-a433-4dd9-9b6d-5d0a410e1a55 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:49.033059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34db3ea-c2ba-4b5a-933f-ee78aac58d5e None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:49.060609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3517d1-b052-4185-9199-3351e730753d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:49.061259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3517d1-b052-4185-9199-3351e730753d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:49.061544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3517d1-b052-4185-9199-3351e730753d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:49.061847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3517d1-b052-4185-9199-3351e730753d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:49.063006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3517d1-b052-4185-9199-3351e730753d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:49.064682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3517d1-b052-4185-9199-3351e730753d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:49.066177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 153/309] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:49.067414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25f00-0a24-4878-acdf-033841404f0d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:49.067694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25f00-0a24-4878-acdf-033841404f0d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:49.067967 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25f00-0a24-4878-acdf-033841404f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:49.077782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:49.078366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:49.079524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:49.079524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:49.079975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] RBAC: Adding query filter params (role.id=24a475c8933346c6a1fa3359fbe0c818, user.id=9434caec0e9e48f280f16bacbd50d865, scope.system=all) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:49.080286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:49.081715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:49.093539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-53a25f00-0a24-4878-acdf-033841404f0d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:49.095058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-53a25f00-0a24-4878-acdf-033841404f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:49.098128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 157/310] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Sep 19 11:43:48 2022] GET /identity/v3/role_assignments?user.id=9434caec0e9e48f280f16bacbd50d865&scope.system=all&role.id=24a475c8933346c6a1fa3359fbe0c818 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:49.099812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:49.111598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:49.163152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:49.208934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 154/311] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:49.250005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64753d-7662-45b9-9f0c-c4ce83f3d3e7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:49.288659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:49.289588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:49.289900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:49.290176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] PATH_INFO: `/v3/groups` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:49.290812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:49.292231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:49.312153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 158/312] 10.222.0.88 () {60 vars in 1134 bytes} [Mon Sep 19 11:43:49 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:43:49.487949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25f00-0a24-4878-acdf-033841404f0d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:49.593916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25f00-0a24-4878-acdf-033841404f0d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:49.630756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:49.631459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:49.631747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:49.632029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:49.643187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:49.648157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:49.649048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:49.649905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 155/313] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:49.658085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:49.728941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:49.767528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19966f-05c2-4899-973f-1960d46869b8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:49.808251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:49.808896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:49.809181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:49.809451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:49.810158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:49.810467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:49.811856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:49.820908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 159/314] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:49.827872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:49.865529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:49.955900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d57c-f4a8-44ff-8adb-d44eaa45a1a0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:49.991972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:49.992905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:49.993335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:49.993807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] PATH_INFO: `/v3/users/7d29e8b91fdf4d50be02c31aab2c20c9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:49.995120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] RBAC: Authorizing `identity:get_user(user_id=7d29e8b91fdf4d50be02c31aab2c20c9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:49.996738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:49.998870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 156/315] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:49 2022] GET /identity/v3/users/7d29e8b91fdf4d50be02c31aab2c20c9 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:50.004882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:50.113516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ea8ac-e255-48c1-a0d4-38d5f826ce81 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:50.142332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:50.154557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:50.155587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:50.156063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:50.156551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:50.232618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:50.234123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:50.337815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 160/316] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:50.343835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:50.454712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:50.457494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea31cb-c0a1-4532-a466-e7b9e8a5db00 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:50.494171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b107c1-4a78-4096-a26a-f1b06521708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:50.494803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b107c1-4a78-4096-a26a-f1b06521708e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:50.495092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b107c1-4a78-4096-a26a-f1b06521708e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:50.495428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b107c1-4a78-4096-a26a-f1b06521708e None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:50.497064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b107c1-4a78-4096-a26a-f1b06521708e None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:50.498542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b107c1-4a78-4096-a26a-f1b06521708e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:50.500541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 157/317] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:50 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:50.506462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:50.535173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:50.608842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4048f6-18c4-41be-8590-305d370190dd None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:50.644274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8deb95-2f46-4b0d-93f7-e8ec8cddee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:50.645215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8deb95-2f46-4b0d-93f7-e8ec8cddee45 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:50.645697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8deb95-2f46-4b0d-93f7-e8ec8cddee45 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:50.646200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8deb95-2f46-4b0d-93f7-e8ec8cddee45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:50.647176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8deb95-2f46-4b0d-93f7-e8ec8cddee45 None admin] RBAC: Adding query filter params (role.id=24a475c8933346c6a1fa3359fbe0c818, scope.project.id=67493e851cd54135a5319695e18189a1, user.id=7d29e8b91fdf4d50be02c31aab2c20c9) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:50.647684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8deb95-2f46-4b0d-93f7-e8ec8cddee45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:50.649320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8deb95-2f46-4b0d-93f7-e8ec8cddee45 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:50.656546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 161/318] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:43:50 2022] GET /identity/v3/role_assignments?user.id=7d29e8b91fdf4d50be02c31aab2c20c9&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=24a475c8933346c6a1fa3359fbe0c818 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:51.146863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19c37-a881-495a-af7e-9afb56a2cd89 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.147289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19c37-a881-495a-af7e-9afb56a2cd89 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.147289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19c37-a881-495a-af7e-9afb56a2cd89 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.148740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 158/319] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:51.154310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74dc5e-80c5-4e0d-b6cb-7b454a305c2e None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.154714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74dc5e-80c5-4e0d-b6cb-7b454a305c2e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.154959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74dc5e-80c5-4e0d-b6cb-7b454a305c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.173198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fb74dc5e-80c5-4e0d-b6cb-7b454a305c2e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:51.174395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fb74dc5e-80c5-4e0d-b6cb-7b454a305c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:51.240397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 162/320] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:51.374219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194d87-e8e3-43f1-9385-a05f45330529 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.374576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194d87-e8e3-43f1-9385-a05f45330529 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.374769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194d87-e8e3-43f1-9385-a05f45330529 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.376254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 159/321] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:51.381383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a30bf7-c7cc-4f6a-9385-cbbc0320311b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.381670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a30bf7-c7cc-4f6a-9385-cbbc0320311b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.381985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a30bf7-c7cc-4f6a-9385-cbbc0320311b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.400921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-56a30bf7-c7cc-4f6a-9385-cbbc0320311b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:51.402061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-56a30bf7-c7cc-4f6a-9385-cbbc0320311b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:51.419437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:51.473083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 163/322] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:51.506164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65194d87-e8e3-43f1-9385-a05f45330529 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:51.611089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65194d87-e8e3-43f1-9385-a05f45330529 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:51.648812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c5182-dec2-46c8-99bc-92dc4165b609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:51.649462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c5182-dec2-46c8-99bc-92dc4165b609 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.649744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c5182-dec2-46c8-99bc-92dc4165b609 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.650053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c5182-dec2-46c8-99bc-92dc4165b609 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.662945 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c5182-dec2-46c8-99bc-92dc4165b609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a62c5182-dec2-46c8-99bc-92dc4165b609 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:51.667722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:51.669074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:51.669919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:51.675404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 160/323] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:51.675556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a30bf7-c7cc-4f6a-9385-cbbc0320311b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:51.706905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d795a14-a192-4907-ba56-e0f43104d299 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.707221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d795a14-a192-4907-ba56-e0f43104d299 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.707469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d795a14-a192-4907-ba56-e0f43104d299 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.708806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 161/324] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:51.714052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947a8f9-9fa2-4395-974e-25f691736588 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.714336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947a8f9-9fa2-4395-974e-25f691736588 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.714663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947a8f9-9fa2-4395-974e-25f691736588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.738415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3947a8f9-9fa2-4395-974e-25f691736588 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:51.740319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3947a8f9-9fa2-4395-974e-25f691736588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:51.774287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:51.787873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a30bf7-c7cc-4f6a-9385-cbbc0320311b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:51.818511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 162/325] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:43:51.819368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947a8f9-9fa2-4395-974e-25f691736588 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:51.825962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:51.826576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.826854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.827125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.827823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:51.828207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:51.829867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:51.842825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 164/326] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:51.848020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:51.864249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:51.928772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947a8f9-9fa2-4395-974e-25f691736588 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:51.961521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9455c1-b130-4bf9-b021-575b4b7b15fc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:51.964190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:51.964891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:51.965694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:51.966124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:51.976933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:51.983130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:51.984030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:51.984855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:51.989219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 163/327] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:51.991576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:52.004043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:52.004688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:52.004967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:52.005240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] PATH_INFO: `/v3/users/f0bb01ed7d264537ab955dbb4b9c3219` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:52.006387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] RBAC: Authorizing `identity:get_user(user_id=f0bb01ed7d264537ab955dbb4b9c3219)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:52.007810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:52.009370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 165/328] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:51 2022] GET /identity/v3/users/f0bb01ed7d264537ab955dbb4b9c3219 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:52.013156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:52.015263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:52.103185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:52.104725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b9035-bf77-44f1-aef1-317cafab8a0c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:52.123047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed22ac-7979-4dbe-b875-3b7eb2c05225 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:52.138803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:52.139416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:52.139693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:52.139971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:52.140635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:52.140936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:52.144465 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:52.152628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 164/329] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:52.153255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:52.162010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:52.162621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:52.162894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:52.163159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:52.163880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] RBAC: Adding query filter params (role.id=3530f0bc598248418cc693afa8f4b150, user.id=f0bb01ed7d264537ab955dbb4b9c3219, scope.system=all) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:52.164182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:52.165579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:52.178099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 166/330] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Sep 19 11:43:52 2022] GET /identity/v3/role_assignments?user.id=f0bb01ed7d264537ab955dbb4b9c3219&scope.system=all&role.id=3530f0bc598248418cc693afa8f4b150 => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:52.178714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:52.260894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:52.287405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e58ca8-6d81-4c37-9540-b51c773977b6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:52.305373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66ff1b-c932-4a4d-87ea-0df06fe25b41 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:52.330667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:52.331322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:52.331601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:52.331875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:52.333087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:52.334584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:52.336429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 165/331] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:43:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:52.342035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:52.348679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:52.349317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:52.349593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:52.349893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] PATH_INFO: `/v3/users/9434caec0e9e48f280f16bacbd50d865` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:52.351029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] RBAC: Authorizing `identity:get_user(user_id=9434caec0e9e48f280f16bacbd50d865)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:52.352486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:52.354132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 167/332] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:52 2022] GET /identity/v3/users/9434caec0e9e48f280f16bacbd50d865 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:52.360817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:52.374886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:52.456318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fbfed-8256-412a-b70f-3a106b64e727 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:52.476555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661276-54fb-4e7f-a206-c4be38a044ba None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:52.489905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702ed39-f030-48d8-9a2a-46de47259685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:52.490519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ed39-f030-48d8-9a2a-46de47259685 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:52.490801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ed39-f030-48d8-9a2a-46de47259685 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:52.491073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ed39-f030-48d8-9a2a-46de47259685 None admin] PATH_INFO: `/v3/groups` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:52.491702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702ed39-f030-48d8-9a2a-46de47259685 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:52.493121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702ed39-f030-48d8-9a2a-46de47259685 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:52.508903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 166/333] 10.222.0.88 () {60 vars in 1134 bytes} [Mon Sep 19 11:43:52 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:43:52.511401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f48498-e32d-495a-92e5-ef6c85c8f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:52.512041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f48498-e32d-495a-92e5-ef6c85c8f3eb None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:52.512321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f48498-e32d-495a-92e5-ef6c85c8f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:52.512595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f48498-e32d-495a-92e5-ef6c85c8f3eb None admin] PATH_INFO: `/v3/system/users/9434caec0e9e48f280f16bacbd50d865/roles/24a475c8933346c6a1fa3359fbe0c818` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:52.514110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f48498-e32d-495a-92e5-ef6c85c8f3eb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9434caec0e9e48f280f16bacbd50d865, role_id=24a475c8933346c6a1fa3359fbe0c818)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:52.515510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f48498-e32d-495a-92e5-ef6c85c8f3eb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:52.523535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 168/334] 10.222.0.88 () {58 vars in 1334 bytes} [Mon Sep 19 11:43:52 2022] PUT /identity/v3/system/users/9434caec0e9e48f280f16bacbd50d865/roles/24a475c8933346c6a1fa3359fbe0c818 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:43:54.967249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7016-8a85-4b9c-8f1f-1ecd25bfe99b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:54.967991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7016-8a85-4b9c-8f1f-1ecd25bfe99b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:54.967991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7016-8a85-4b9c-8f1f-1ecd25bfe99b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:54.969505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 167/335] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:54.975064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63666d3-dcb0-42f9-91b6-16ba3ddb4776 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:54.975743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63666d3-dcb0-42f9-91b6-16ba3ddb4776 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:54.975743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63666d3-dcb0-42f9-91b6-16ba3ddb4776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:54.999912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e63666d3-dcb0-42f9-91b6-16ba3ddb4776 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:55.001193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e63666d3-dcb0-42f9-91b6-16ba3ddb4776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:55.085820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 169/336] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:55.535757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d23bba-cb8b-41ed-81df-173740394c1b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:55.536129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d23bba-cb8b-41ed-81df-173740394c1b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:55.536323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d23bba-cb8b-41ed-81df-173740394c1b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:55.537823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 168/337] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:55.555904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1854f-d7d3-481a-bebc-264b12b048a9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:55.556242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1854f-d7d3-481a-bebc-264b12b048a9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:55.556454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1854f-d7d3-481a-bebc-264b12b048a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:55.563091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:55.580006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fbe1854f-d7d3-481a-bebc-264b12b048a9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:55.581233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fbe1854f-d7d3-481a-bebc-264b12b048a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:55.635999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d23bba-cb8b-41ed-81df-173740394c1b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:55.659660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:55.675723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 170/338] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:55.695824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b563c6f-fe7d-4221-8fd1-d5e89ea4a49b None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:55.696373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b563c6f-fe7d-4221-8fd1-d5e89ea4a49b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:55.696884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b563c6f-fe7d-4221-8fd1-d5e89ea4a49b None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:55.698659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 171/339] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:43:55.707311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c999-5523-44be-9f0e-5db3bef743c2 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:55.707822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c999-5523-44be-9f0e-5db3bef743c2 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:55.708239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82c999-5523-44be-9f0e-5db3bef743c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:55.740519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fe82c999-5523-44be-9f0e-5db3bef743c2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:55.742023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fe82c999-5523-44be-9f0e-5db3bef743c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:55.846214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d23bba-cb8b-41ed-81df-173740394c1b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:55.885526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 172/340] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:55.903448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:55.904133 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:55.904414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:55.904691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:55.924894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:55.930370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:55.931302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:55.932147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 169/341] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:55.937492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82c999-5523-44be-9f0e-5db3bef743c2 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:56.000408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:56.026492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:56.080418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82c999-5523-44be-9f0e-5db3bef743c2 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:56.119431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:56.121159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34715db-054a-45a7-89e0-eeeaa2c52a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:56.122570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34715db-054a-45a7-89e0-eeeaa2c52a11 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.123002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34715db-054a-45a7-89e0-eeeaa2c52a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.123282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34715db-054a-45a7-89e0-eeeaa2c52a11 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.123962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34715db-054a-45a7-89e0-eeeaa2c52a11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:56.124505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34715db-054a-45a7-89e0-eeeaa2c52a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:56.132378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34715db-054a-45a7-89e0-eeeaa2c52a11 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:56.149608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 173/342] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:56.151402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2653-4fbf-40a8-acdf-4b2e503042fe None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.151960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2653-4fbf-40a8-acdf-4b2e503042fe None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.152458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2653-4fbf-40a8-acdf-4b2e503042fe None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.154016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 174/343] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:56.159642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e0eac-3d1f-4ade-84b1-972d0104f49c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.160296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e0eac-3d1f-4ade-84b1-972d0104f49c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.160724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e0eac-3d1f-4ade-84b1-972d0104f49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.181166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c90e0eac-3d1f-4ade-84b1-972d0104f49c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:56.183167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c90e0eac-3d1f-4ade-84b1-972d0104f49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:56.189043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b86506-24b8-4910-a2e6-68ab6011f0d9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:56.247669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:56.248579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.249577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.250788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.274499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:56.285390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:56.286657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:43:56.287737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:56.291354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 170/344] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:43:56.293605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:56.316860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 175/345] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:56.318038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e0eac-3d1f-4ade-84b1-972d0104f49c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:56.432360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:56.490132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d0319-9fcf-45e9-a2e4-be1101c7f207 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:56.526307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e0eac-3d1f-4ade-84b1-972d0104f49c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:56.530056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:56.530685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.530974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.531249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.533684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:56.582082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:56.582703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.582982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.583254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.583923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:56.584401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:56.594001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:56.614445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:56.616206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 176/346] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:56.616370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:56.617057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:56.740216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 171/347] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:56.741506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:56.774145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:56.778645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91fc2d-36d8-42a5-9683-d67300c86778 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:56.798186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:56.826896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:56.827861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.828736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.829160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.842713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:56.848621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:56.849704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:43:56.850572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:56.853650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 177/348] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:56.854686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:56.880151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0aa27-76e3-4095-95a1-a2907de292c2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:56.934987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:56.936249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:56.937079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:56.938392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] PATH_INFO: `/v3/users/9434caec0e9e48f280f16bacbd50d865` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:56.940266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] RBAC: Authorizing `identity:get_user(user_id=9434caec0e9e48f280f16bacbd50d865)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:56.942476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:56.944893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 172/349] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:43:56 2022] GET /identity/v3/users/9434caec0e9e48f280f16bacbd50d865 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:56.945856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.002932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c33ee1-02d1-431f-b4da-414985ed0d73 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.044949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.045739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.046080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.046354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.060090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:57.065199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:57.066333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:57.067369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:57.069530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 178/350] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:57.070554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.078216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:57.087705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27af84-13df-408c-904b-4e0ffe2851ad None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.113622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:57.138125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.139288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.139627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.139973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.141168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:57.142691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:57.144526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 173/351] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:56 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:57.145110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.200372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ec8c4-b96f-45a6-9ca9-770a43d1370d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.251921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444972a-e7d0-4d79-931e-50d9334c8b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.252642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444972a-e7d0-4d79-931e-50d9334c8b79 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.252919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444972a-e7d0-4d79-931e-50d9334c8b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.253238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444972a-e7d0-4d79-931e-50d9334c8b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.253892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444972a-e7d0-4d79-931e-50d9334c8b79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:57.254193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444972a-e7d0-4d79-931e-50d9334c8b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:57.255559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444972a-e7d0-4d79-931e-50d9334c8b79 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:57.266429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 179/352] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:43:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:57.267698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b68b04-2a30-48aa-b43f-5d186bbc778c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.267980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b68b04-2a30-48aa-b43f-5d186bbc778c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.268249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b68b04-2a30-48aa-b43f-5d186bbc778c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.269853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 180/353] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:57.270530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b68b04-2a30-48aa-b43f-5d186bbc778c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.290708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d536e-50b0-4794-9e2f-fb1d6343c95a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.337715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.338538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.338816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.339087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.340418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] RBAC: Adding query filter params (role.id=24a475c8933346c6a1fa3359fbe0c818, user.id=9434caec0e9e48f280f16bacbd50d865, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:57.340744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:57.342229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:57.356063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 174/354] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Sep 19 11:43:57 2022] GET /identity/v3/role_assignments?user.id=9434caec0e9e48f280f16bacbd50d865&scope.system=all&role.id=24a475c8933346c6a1fa3359fbe0c818 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:57.356063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.396720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:57.418614 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:57.427096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b68b04-2a30-48aa-b43f-5d186bbc778c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.476611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31cc48-b8dc-4784-bfce-ffc9e1b4a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.477229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31cc48-b8dc-4784-bfce-ffc9e1b4a429 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.477503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31cc48-b8dc-4784-bfce-ffc9e1b4a429 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.477792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31cc48-b8dc-4784-bfce-ffc9e1b4a429 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.478605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31cc48-b8dc-4784-bfce-ffc9e1b4a429 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:57.478913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31cc48-b8dc-4784-bfce-ffc9e1b4a429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:57.480679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31cc48-b8dc-4784-bfce-ffc9e1b4a429 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:57.491854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 181/355] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:57.493365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e67f-961d-450a-8f76-4e70396face8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.493365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e67f-961d-450a-8f76-4e70396face8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.494611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e67f-961d-450a-8f76-4e70396face8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.505380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9c6a0-a2a3-45a0-a867-45c641621437 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.516840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-eb87e67f-961d-450a-8f76-4e70396face8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:57.518639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-eb87e67f-961d-450a-8f76-4e70396face8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:57.551487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.552144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.552425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.552698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.560482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:57.562323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:57.563480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 175/356] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:43:57 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:57.564108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.611778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 182/357] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:57.612456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e67f-961d-450a-8f76-4e70396face8 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.675979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:57.704997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7245c-9358-4ea8-a669-ee31fced3eed None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.735568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:57.747982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40e5c6-e06d-4713-9a29-9e37435bd22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.748593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40e5c6-e06d-4713-9a29-9e37435bd22b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.748866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40e5c6-e06d-4713-9a29-9e37435bd22b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.749134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40e5c6-e06d-4713-9a29-9e37435bd22b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.751226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e67f-961d-450a-8f76-4e70396face8 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:57.794504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:57.795124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.795408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.795790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.842056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40e5c6-e06d-4713-9a29-9e37435bd22b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:57.843762 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40e5c6-e06d-4713-9a29-9e37435bd22b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:57.880111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:57.881537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:57.939511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:57.957790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 176/358] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:57.959048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4a16-935a-4ecb-88b6-425301128bc8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:57.962282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4a16-935a-4ecb-88b6-425301128bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:57.962282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4a16-935a-4ecb-88b6-425301128bc8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:57.962282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 177/359] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:43:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:43:57.962282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c4a16-935a-4ecb-88b6-425301128bc8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:57.990104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:58.001803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 183/360] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:58.002665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.083940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c4a16-935a-4ecb-88b6-425301128bc8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.123837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2de21-42e3-4510-84b3-46ee14502212 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.125051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.125659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.125965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.126240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.126975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=eeb8255bb2f340b6aad60728a547a86c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:58.127288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:58.128745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:58.136454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 178/361] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:43:57 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=3530f0bc598248418cc693afa8f4b150 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:58.138053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.170474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fe9ce-c18c-4cd2-a041-df9e36ed962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.171118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fe9ce-c18c-4cd2-a041-df9e36ed962b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.171463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fe9ce-c18c-4cd2-a041-df9e36ed962b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.171656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fe9ce-c18c-4cd2-a041-df9e36ed962b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.186427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fe9ce-c18c-4cd2-a041-df9e36ed962b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-7d8fe9ce-c18c-4cd2-a041-df9e36ed962b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:58.191373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:58.192560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:43:58.193487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 184/362] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:58.194855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146282b5-f067-47ef-96e7-5be365d51bf0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.195139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146282b5-f067-47ef-96e7-5be365d51bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.195414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146282b5-f067-47ef-96e7-5be365d51bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.215138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-146282b5-f067-47ef-96e7-5be365d51bf0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:43:58.216354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-146282b5-f067-47ef-96e7-5be365d51bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:58.224371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:58.285374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:58.304902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131a07-16e2-4fcc-b9c5-f936cdee5ad2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.313608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 185/363] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:43:58.313608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146282b5-f067-47ef-96e7-5be365d51bf0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.347480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.348168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.348412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.348684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.349948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:58.351397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:58.353575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 179/364] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:58.354394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.447027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146282b5-f067-47ef-96e7-5be365d51bf0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.486825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cf9f0-69e6-46a5-bb86-27ba0a450ea3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.493268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.493985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.494262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.494537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.495733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:58.497715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:58.499711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 186/365] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:58.500500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.508341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:58.531280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.531994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.532281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.532555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.533323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:58.533633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:58.535148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:58.544147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 180/366] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:58.545187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.559901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:58.617948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d86679-ba5c-4cb6-8269-2c8aff1eac5b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.655826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.656459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.656762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.657033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.658185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:58.660457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:58.662610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 187/367] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:58.663480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.665600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79ead-c891-4dda-b1c7-b59509503f9b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.703978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.704623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.704903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.705174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.708738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:58.708738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:58.709555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 181/368] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:58.710184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.764307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:58.794404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b34a6-9668-4414-9e50-1a6ce1554912 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.815688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:58.831205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fea98-3017-4975-b3cb-558b06cba4a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:58.837151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.837857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.838139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.838661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.873138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:58.873704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:58.874031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:58.874303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:58.891209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:43:58.898418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:43:58.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:43:58.900272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 182/369] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:43:58.901053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:58.938512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:58.940240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:59.059683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51835df-9600-4a3b-a7e1-4ee22f9f7d7f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:59.073099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:59.094814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 188/370] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:59.095900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:59.108459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:59.109153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:59.109498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:59.110228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:59.111228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=67493e851cd54135a5319695e18189a1) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:59.111228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:59.113289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:59.119758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 183/371] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:43:58 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=3530f0bc598248418cc693afa8f4b150 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:59.122436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:59.122574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:59.238883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702afe01-8270-4d72-ad45-66a77189f838 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:59.256488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1d1a9-aa49-4050-b707-d3c67a8db40f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:59.280457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:59.281194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:59.281383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:59.281663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:59.282454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=eeb8255bb2f340b6aad60728a547a86c) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:59.282768 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:59.284226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:59.296083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 189/372] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:43:59 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=0a108a4463a4458480b00aed843b4e9b => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:59.297086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:59.305270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:59.306393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:59.306933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:59.307400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:59.308274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:43:59.309332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:59.311015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:59.322753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 184/373] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:43:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:59.334940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:59.385154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:59.470918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:59.500050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bfd65-0f7f-4ff0-87da-734ff86a4556 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:59.516010 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc90171-0e6b-43b9-980e-7996b6961632 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:59.546395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:59.547189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:59.547465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:59.547734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:59.548872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:59.550397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:59.553239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 190/374] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:59 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:43:59.559911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:59.577116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:59.577747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:59.578055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:59.578333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:59.693493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:59.695000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:59.717874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef31f4-0cc1-405c-88ee-f609db23eded None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:59.766047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:59.792168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:59.792817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:59.793629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:59.793987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:59.795264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:59.796708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:59.800122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 191/375] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:43:59 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:43:59.805002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:59.846142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:59.881954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 185/376] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:43:59.884071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:59.971141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f446e8-d460-4193-b9f3-9c6bf7007bb2 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:00.015851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:00.016836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:00.017122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:00.017445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:00.018512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=67493e851cd54135a5319695e18189a1) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:00.018825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:00.020431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:00.027830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 192/377] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:43:59 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=0a108a4463a4458480b00aed843b4e9b => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:00.028584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e971dd8-c297-49d7-a00d-c9f5dec693a4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:00.076539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:00.077205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:00.077505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:00.077933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] PATH_INFO: `/v3/projects/a73ceb365cbe45598f5feb33f3998086` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:00.080258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] RBAC: Authorizing `identity:get_project(project_id=a73ceb365cbe45598f5feb33f3998086)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:00.082317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:00.084913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 186/378] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:43:59 2022] GET /identity/v3/projects/a73ceb365cbe45598f5feb33f3998086 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:00.090850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:00.195986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:00.220924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0bd81-75aa-4c1f-bc06-42a1c9920c18 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:00.263571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290483d8-4cdf-4d88-b867-07c912c35bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:00.264220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290483d8-4cdf-4d88-b867-07c912c35bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:00.264490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290483d8-4cdf-4d88-b867-07c912c35bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:00.264758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290483d8-4cdf-4d88-b867-07c912c35bc6 None admin] PATH_INFO: `/v3/groups/c6f933d34324493294b2b8092c8e7c7f` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:00.272507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290483d8-4cdf-4d88-b867-07c912c35bc6 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f933d34324493294b2b8092c8e7c7f)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:00.273936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290483d8-4cdf-4d88-b867-07c912c35bc6 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:00.276693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 193/379] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:00 2022] GET /identity/v3/groups/c6f933d34324493294b2b8092c8e7c7f => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:00.282957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:00.286532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:00.410784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a60ad6-c52c-4fb7-94b7-91c7047c4399 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:00.460584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf642d27-0bab-4a0f-be76-bbb13d8c42e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:00.461210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf642d27-0bab-4a0f-be76-bbb13d8c42e3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:00.461541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf642d27-0bab-4a0f-be76-bbb13d8c42e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:00.461841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf642d27-0bab-4a0f-be76-bbb13d8c42e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:00.462649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf642d27-0bab-4a0f-be76-bbb13d8c42e3 None admin] RBAC: Adding query filter params (group.id=c6f933d34324493294b2b8092c8e7c7f, role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=a73ceb365cbe45598f5feb33f3998086) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:00.462966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf642d27-0bab-4a0f-be76-bbb13d8c42e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:00.464405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf642d27-0bab-4a0f-be76-bbb13d8c42e3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:00.471135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 187/380] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:44:00 2022] GET /identity/v3/role_assignments?group.id=c6f933d34324493294b2b8092c8e7c7f&scope.project.id=a73ceb365cbe45598f5feb33f3998086&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:00.976770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecb55d-dce7-4e7f-8615-8aba2ca0de9f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:00.977141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecb55d-dce7-4e7f-8615-8aba2ca0de9f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:00.977333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecb55d-dce7-4e7f-8615-8aba2ca0de9f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:00.979090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 194/381] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:00.994716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26a71-4e46-4170-87e6-2f1958c11e86 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:00.995005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26a71-4e46-4170-87e6-2f1958c11e86 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:00.995272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26a71-4e46-4170-87e6-2f1958c11e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:01.015146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-60f26a71-4e46-4170-87e6-2f1958c11e86 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:01.016340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60f26a71-4e46-4170-87e6-2f1958c11e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:01.080282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:01.088491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 188/382] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:01.396073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecb55d-dce7-4e7f-8615-8aba2ca0de9f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:01.505535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecb55d-dce7-4e7f-8615-8aba2ca0de9f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:01.541517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:01.542192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:01.542475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:01.542747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:01.553655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:01.559067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:01.559999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:01.560957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 195/383] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:01.566141 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f26a71-4e46-4170-87e6-2f1958c11e86 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:01.730728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f26a71-4e46-4170-87e6-2f1958c11e86 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:01.785899 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62ae51-9c11-4156-80e6-85dc3baf107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:01.786520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62ae51-9c11-4156-80e6-85dc3baf107c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:01.786806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62ae51-9c11-4156-80e6-85dc3baf107c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:01.787084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62ae51-9c11-4156-80e6-85dc3baf107c None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:01.787753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62ae51-9c11-4156-80e6-85dc3baf107c None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:01.788056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62ae51-9c11-4156-80e6-85dc3baf107c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:01.789419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62ae51-9c11-4156-80e6-85dc3baf107c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:01.801514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 189/384] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:44:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:01.807038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:01.811592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:02.034285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c422fcf-a932-4db8-8a9d-87493c350242 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.034565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c422fcf-a932-4db8-8a9d-87493c350242 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.035095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c422fcf-a932-4db8-8a9d-87493c350242 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.036630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 190/385] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:02.041709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b7b40-28aa-4dfc-9a33-1c23d13e6652 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:02.043383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bd053-f55a-4fed-b69f-199f033f078b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.043891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bd053-f55a-4fed-b69f-199f033f078b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.044363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bd053-f55a-4fed-b69f-199f033f078b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.075101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d99bd053-f55a-4fed-b69f-199f033f078b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:02.079105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d99bd053-f55a-4fed-b69f-199f033f078b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:02.092684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:02.093339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.093620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.095560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.096861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:02.098325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:02.105808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 196/386] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:01 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:02.111952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:02.197979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:02.226808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 191/387] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:02.274224 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:02.295262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d5b64-6178-4d60-8061-70ebe167899e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:02.329970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992849b-2645-4957-8865-4fecfbb0ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:02.330609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992849b-2645-4957-8865-4fecfbb0ed3a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.330882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992849b-2645-4957-8865-4fecfbb0ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.331153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992849b-2645-4957-8865-4fecfbb0ed3a None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.332303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992849b-2645-4957-8865-4fecfbb0ed3a None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:02.333803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992849b-2645-4957-8865-4fecfbb0ed3a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:02.335589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 197/388] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:02 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:02.342164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bd053-f55a-4fed-b69f-199f033f078b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:02.357816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e0d95-3865-4929-9c6e-b120a0b17cae None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.358114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e0d95-3865-4929-9c6e-b120a0b17cae None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.358359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e0d95-3865-4929-9c6e-b120a0b17cae None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.363012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 198/389] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:02.365657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea51f-345e-4c2a-a132-bff4480b0d48 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.365960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea51f-345e-4c2a-a132-bff4480b0d48 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.366230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea51f-345e-4c2a-a132-bff4480b0d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.398853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-838ea51f-345e-4c2a-a132-bff4480b0d48 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:02.400084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-838ea51f-345e-4c2a-a132-bff4480b0d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:02.509923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bd053-f55a-4fed-b69f-199f033f078b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:02.538707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 199/390] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:02.539972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ea51f-345e-4c2a-a132-bff4480b0d48 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:02.561222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:02.562127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.562239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.562497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c/groups/718045f856084708a32bc015147ad1e8/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.564474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb8255bb2f340b6aad60728a547a86c, group_id=718045f856084708a32bc015147ad1e8, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:02.566248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:02.583774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 192/391] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Sep 19 11:44:02 2022] PUT /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c/groups/718045f856084708a32bc015147ad1e8/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:02.631424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:02.669206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ea51f-345e-4c2a-a132-bff4480b0d48 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:02.718636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e63ed6-b8a3-4d9b-a1ce-6332306b46a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:02.719306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e63ed6-b8a3-4d9b-a1ce-6332306b46a0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.719582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e63ed6-b8a3-4d9b-a1ce-6332306b46a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.719855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e63ed6-b8a3-4d9b-a1ce-6332306b46a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.737471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e63ed6-b8a3-4d9b-a1ce-6332306b46a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-14e63ed6-b8a3-4d9b-a1ce-6332306b46a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:02.742519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:02.743484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:02.744306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 200/392] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:02.751438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:02.813460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:02.826397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444f969-d15b-4145-bbc5-5600053490ef None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.826969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444f969-d15b-4145-bbc5-5600053490ef None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.827248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444f969-d15b-4145-bbc5-5600053490ef None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.830242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 201/393] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:02.835530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a9612-9560-44eb-86ff-f357df0bcb3d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.835812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a9612-9560-44eb-86ff-f357df0bcb3d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.836093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a9612-9560-44eb-86ff-f357df0bcb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.855082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b04a9612-9560-44eb-86ff-f357df0bcb3d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:02.856324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b04a9612-9560-44eb-86ff-f357df0bcb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:02.878367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a0487-2953-4b44-8fc5-341d764a952a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:02.935501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 202/394] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:02.954500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:02.955137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:02.955486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:02.955766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:02.956446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:02.956751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:02.958155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:02.977238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 193/395] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:44:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:02.983212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a9612-9560-44eb-86ff-f357df0bcb3d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:02.987461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.076612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:03.110735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305651d-523e-41b9-83d4-87c9c84d75a7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.124271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:03.157549 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a9612-9560-44eb-86ff-f357df0bcb3d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.170870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cd014-47c9-4d3e-a23b-fc8d83389b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.172493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cd014-47c9-4d3e-a23b-fc8d83389b04 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.173211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cd014-47c9-4d3e-a23b-fc8d83389b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.174167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cd014-47c9-4d3e-a23b-fc8d83389b04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.198102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cd014-47c9-4d3e-a23b-fc8d83389b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-170cd014-47c9-4d3e-a23b-fc8d83389b04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:03.207445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:03.208409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:03.209257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:03.213495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 194/396] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:03.220676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b15739-2a0d-40e3-8183-6b55e462f028 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.220956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b15739-2a0d-40e3-8183-6b55e462f028 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.221221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b15739-2a0d-40e3-8183-6b55e462f028 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.222920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 195/397] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:03.224469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b15739-2a0d-40e3-8183-6b55e462f028 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.227280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a632a3b-268b-4f84-b6cf-073624980758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.228153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632a3b-268b-4f84-b6cf-073624980758 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.228635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632a3b-268b-4f84-b6cf-073624980758 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.229108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632a3b-268b-4f84-b6cf-073624980758 None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.231452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a632a3b-268b-4f84-b6cf-073624980758 None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:03.233134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a632a3b-268b-4f84-b6cf-073624980758 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:03.235234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 203/398] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:02 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:03.236579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab54be-4903-4f8b-a05a-c8b0eda8a3bf None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.237067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab54be-4903-4f8b-a05a-c8b0eda8a3bf None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.237552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab54be-4903-4f8b-a05a-c8b0eda8a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.258592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-23ab54be-4903-4f8b-a05a-c8b0eda8a3bf None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:03.260337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-23ab54be-4903-4f8b-a05a-c8b0eda8a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:03.344969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 204/399] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:03.345843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab54be-4903-4f8b-a05a-c8b0eda8a3bf None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.360747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b15739-2a0d-40e3-8183-6b55e462f028 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.406280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.406280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.406280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.406641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.407228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:03.407572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:03.409280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:03.420281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 196/400] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:03.421135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.426435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:03.466542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:03.489822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab54be-4903-4f8b-a05a-c8b0eda8a3bf None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.530085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.530871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.531240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.531606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.532852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:03.534624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:03.541248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 205/401] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:03 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:03.542239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.555663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75eded-9e0d-4f6e-b7f7-9a9146f4df51 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.608479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059961f-8579-4dd1-b653-136068e800df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.610505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059961f-8579-4dd1-b653-136068e800df None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.611055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059961f-8579-4dd1-b653-136068e800df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.611784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059961f-8579-4dd1-b653-136068e800df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.626689 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059961f-8579-4dd1-b653-136068e800df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9059961f-8579-4dd1-b653-136068e800df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:03.635336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:03.636555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:03.637817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:03.640618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 197/402] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:03.641528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059961f-8579-4dd1-b653-136068e800df None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.665049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f28b5-1b86-4391-af2d-d350a2416165 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.695508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:03.706710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.707646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.708074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.708492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.709801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:03.711638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:03.713671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 206/403] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:03 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:03.715217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.742178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:03.754236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059961f-8579-4dd1-b653-136068e800df None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.798185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.798844 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.799191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.799392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1/groups/718045f856084708a32bc015147ad1e8/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.801279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] RBAC: Authorizing `identity:create_grant(project_id=67493e851cd54135a5319695e18189a1, group_id=718045f856084708a32bc015147ad1e8, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:03.802834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:03.816252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 198/404] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Sep 19 11:44:03 2022] PUT /identity/v3/projects/67493e851cd54135a5319695e18189a1/groups/718045f856084708a32bc015147ad1e8/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:03.817458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.863301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4ef26-8f6d-4c9c-a1bc-f1d129d6cd45 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:03.910579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:03.911237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:03.911518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:03.911790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:03.927447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:03.934792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:03.935816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:03.936735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 207/405] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:03.937424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:03.985660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586863f-1d8c-4a63-83ac-3a4edb932d4e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.015534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:04.025139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.026176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.026676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.027727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.028625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:04.029135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.030742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.040598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 199/406] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:04.041747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.060622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:04.072851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95635fa4-9843-4ca0-8e28-3a6b6f35acbe None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.117834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.118394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.118678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.118956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.120677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.122509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.125106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 208/407] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:03 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:04.130524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.202405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958744-8ead-45b5-858a-08c7fe178b98 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.253431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.254482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.255591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.256071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.257207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:04.257986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.260312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.268977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c84b-4fd6-4fa5-b87e-4629e08e10a5 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.273575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 200/408] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:04.274692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.301266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:04.314455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.315609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.316088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.316582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.317943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.319806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.322024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 209/409] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:04 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:04.323083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.378610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:04.404961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a86b1-6b6d-4227-bf99-9e5a0bcb139e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.442633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.443682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.444163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.444637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c/groups/718045f856084708a32bc015147ad1e8/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.447425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb8255bb2f340b6aad60728a547a86c, group_id=718045f856084708a32bc015147ad1e8, role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.449649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.462303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567285e-d2dc-4a2e-a3ae-f4480d1c694f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.472801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 201/410] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Sep 19 11:44:04 2022] PUT /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c/groups/718045f856084708a32bc015147ad1e8/roles/0a108a4463a4458480b00aed843b4e9b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:04.472801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.564577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.566451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.566734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.567011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] PATH_INFO: `/v3/groups/c6f933d34324493294b2b8092c8e7c7f` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.568362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f933d34324493294b2b8092c8e7c7f)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.577967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.580099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 210/411] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:04 2022] GET /identity/v3/groups/c6f933d34324493294b2b8092c8e7c7f => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:04.587144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.625294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67d122-ce12-4684-b59a-a4971c390b49 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.698539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.699661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.699947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.700223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.702840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.705279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.709826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 202/412] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:04 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:04.711443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:04.718886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.724230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:04.771326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e5a0e-cc91-49aa-8fc8-0f46e5dba0d9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.851760 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.852418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.852700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.852980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] PATH_INFO: `/v3/projects/a73ceb365cbe45598f5feb33f3998086` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.855813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] RBAC: Authorizing `identity:get_project(project_id=a73ceb365cbe45598f5feb33f3998086)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.857262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.860414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 211/413] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:04 2022] GET /identity/v3/projects/a73ceb365cbe45598f5feb33f3998086 => generated 344 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:04.867256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:04.886473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db04a25-b9eb-458d-b7a2-f4be8138fec5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:04.967153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a49b8-f0d4-402a-8b0e-6497a654018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:04.967791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a49b8-f0d4-402a-8b0e-6497a654018a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:04.968171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a49b8-f0d4-402a-8b0e-6497a654018a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:04.968590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a49b8-f0d4-402a-8b0e-6497a654018a None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1/groups/718045f856084708a32bc015147ad1e8/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:04.972672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a49b8-f0d4-402a-8b0e-6497a654018a None admin] RBAC: Authorizing `identity:create_grant(project_id=67493e851cd54135a5319695e18189a1, group_id=718045f856084708a32bc015147ad1e8, role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:04.974143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a49b8-f0d4-402a-8b0e-6497a654018a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:04.996725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 203/414] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Sep 19 11:44:04 2022] PUT /identity/v3/projects/67493e851cd54135a5319695e18189a1/groups/718045f856084708a32bc015147ad1e8/roles/0a108a4463a4458480b00aed843b4e9b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:44:05.048998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca18163-bc22-4bb0-a47b-dc858e399b90 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:05.051407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848cae-cdfc-4c01-80c9-9f826f4dae3b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:05.051690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848cae-cdfc-4c01-80c9-9f826f4dae3b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:05.051954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38848cae-cdfc-4c01-80c9-9f826f4dae3b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:05.053318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 204/415] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:05.060871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb28f5-a11a-4e70-9bdc-127fb81c31ac None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:05.061173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb28f5-a11a-4e70-9bdc-127fb81c31ac None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:05.061412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb28f5-a11a-4e70-9bdc-127fb81c31ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:05.090895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-80fb28f5-a11a-4e70-9bdc-127fb81c31ac None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:05.092135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-80fb28f5-a11a-4e70-9bdc-127fb81c31ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:05.118220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:05.137646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:05.140387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:05.140802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:05.141199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] PATH_INFO: `/v3/projects/a73ceb365cbe45598f5feb33f3998086/groups/c6f933d34324493294b2b8092c8e7c7f/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:05.156089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a73ceb365cbe45598f5feb33f3998086, group_id=c6f933d34324493294b2b8092c8e7c7f, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:05.156089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:05.177901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 212/416] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Sep 19 11:44:04 2022] PUT /identity/v3/projects/a73ceb365cbe45598f5feb33f3998086/groups/c6f933d34324493294b2b8092c8e7c7f/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:05.218237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:05.228127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 205/417] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 11:44:05.511830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:05.620709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3333d-da86-4251-8b33-6d52d43c97e1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:05.657298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:05.657970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:05.658246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:05.658516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:05.669387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:05.674610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:05.675525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:05.676353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 213/418] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:05.681930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb28f5-a11a-4e70-9bdc-127fb81c31ac None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:05.791776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb28f5-a11a-4e70-9bdc-127fb81c31ac None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:05.844399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:05.845016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:05.845271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:05.845550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:05.846245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:05.846546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:05.847977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:05.861572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 206/419] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:44:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:05.867240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:05.947100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:06.019085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d241f29-0ddd-49b1-90bd-9ae3b4d45631 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:06.065275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5edda-2f2c-4e3e-b716-96f7ada8d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:06.065839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5edda-2f2c-4e3e-b716-96f7ada8d0ff None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:06.066124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5edda-2f2c-4e3e-b716-96f7ada8d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:06.066399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5edda-2f2c-4e3e-b716-96f7ada8d0ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:06.169892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5edda-2f2c-4e3e-b716-96f7ada8d0ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:06.171348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5edda-2f2c-4e3e-b716-96f7ada8d0ff None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:06.240714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:06.317607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 214/420] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:06.322751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:06.383126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e230-f201-4cfd-a0b9-3acbb2731e36 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:06.383484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e230-f201-4cfd-a0b9-3acbb2731e36 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:06.383838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e230-f201-4cfd-a0b9-3acbb2731e36 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:06.385803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 215/421] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:06.390669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b9d0-1798-4fd4-a95f-23b68f6e99b3 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:06.391432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b9d0-1798-4fd4-a95f-23b68f6e99b3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:06.391924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b9d0-1798-4fd4-a95f-23b68f6e99b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:06.416266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3b42b9d0-1798-4fd4-a95f-23b68f6e99b3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:06.419376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3b42b9d0-1798-4fd4-a95f-23b68f6e99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:06.424052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:06.512453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e47a6-127c-4639-b7ef-065cad557633 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:06.561866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 216/422] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 11:44:06.618599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:06.619172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:06.619458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:06.619738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:06.620974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:06.622452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:06.624398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 207/423] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:06 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:06.629572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b9d0-1798-4fd4-a95f-23b68f6e99b3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:06.757814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:06.789469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b9d0-1798-4fd4-a95f-23b68f6e99b3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:06.826900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:06.827937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:06.828712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:06.829197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:06.830629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:06.832283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:06.834454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 217/424] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:06 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:06.842860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:07.002411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:07.013787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f6167-dc72-4335-9f86-dc4fad61588a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:07.046270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:07.063790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:07.064401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.064681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.064954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.065689 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=eeb8255bb2f340b6aad60728a547a86c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:07.066027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:07.067456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:07.084335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 208/425] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:44:06 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=3530f0bc598248418cc693afa8f4b150 => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:07.154065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813c649-f7e2-4075-9ea9-9eb2d6301a0e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:07.209189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:07.209884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.210170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.210449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.228643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:07.234900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:07.235919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:07.236754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 218/426] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:07.241855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:07.369706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bcc20-60a3-44f4-a9f6-b91efe6695f1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:07.405890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6c392-2f84-4d8f-a73f-d6c253e302e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:07.406493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c392-2f84-4d8f-a73f-d6c253e302e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.406769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c392-2f84-4d8f-a73f-d6c253e302e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.407037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c392-2f84-4d8f-a73f-d6c253e302e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.407699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6c392-2f84-4d8f-a73f-d6c253e302e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:07.407999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6c392-2f84-4d8f-a73f-d6c253e302e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:07.409353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6c392-2f84-4d8f-a73f-d6c253e302e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:07.421190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 209/427] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:44:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:07.426988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:07.437102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:07.470197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a88ffc-c6e6-41d2-95dc-063f4eb46bc3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.470507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a88ffc-c6e6-41d2-95dc-063f4eb46bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.470741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a88ffc-c6e6-41d2-95dc-063f4eb46bc3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.472057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 210/428] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:07.479240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb777-99b4-4d87-ae7f-582cfde3477e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.479517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb777-99b4-4d87-ae7f-582cfde3477e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.479796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb777-99b4-4d87-ae7f-582cfde3477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.503031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-993eb777-99b4-4d87-ae7f-582cfde3477e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:07.504267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-993eb777-99b4-4d87-ae7f-582cfde3477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:07.546910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c01a0-ff85-4251-867e-14ecbb851973 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:07.566515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:07.585632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2f037-aa82-4151-9c62-c585a09246bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:07.586963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2f037-aa82-4151-9c62-c585a09246bf None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2f037-aa82-4151-9c62-c585a09246bf None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.587860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2f037-aa82-4151-9c62-c585a09246bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.595773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 211/429] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:07.675151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2f037-aa82-4151-9c62-c585a09246bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:07.676511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2f037-aa82-4151-9c62-c585a09246bf None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:07.679355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:07.794358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 219/430] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:07.800080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb777-99b4-4d87-ae7f-582cfde3477e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:07.828137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640afc-1a41-4813-ba88-c325a85ca8c9 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.828705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640afc-1a41-4813-ba88-c325a85ca8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.828936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640afc-1a41-4813-ba88-c325a85ca8c9 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.830619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 220/431] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:07.836798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ceab6-d280-4085-b870-8a4764f5c16c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:07.837109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ceab6-d280-4085-b870-8a4764f5c16c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:07.837743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ceab6-d280-4085-b870-8a4764f5c16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:07.864706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d07ceab6-d280-4085-b870-8a4764f5c16c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:07.866559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d07ceab6-d280-4085-b870-8a4764f5c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:07.946594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb777-99b4-4d87-ae7f-582cfde3477e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:07.951038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 221/432] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:07.951991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ceab6-d280-4085-b870-8a4764f5c16c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:07.995271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:07.999709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0de8d-46de-4db7-808c-0be96cd5b962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.000200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0de8d-46de-4db7-808c-0be96cd5b962 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.000479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0de8d-46de-4db7-808c-0be96cd5b962 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.000747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0de8d-46de-4db7-808c-0be96cd5b962 None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.002620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0de8d-46de-4db7-808c-0be96cd5b962 None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:08.003483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0de8d-46de-4db7-808c-0be96cd5b962 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:08.005175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 212/433] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:07 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:08.010082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ec88-2313-42ee-b9d0-3afe9d30cdb4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.010082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ec88-2313-42ee-b9d0-3afe9d30cdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.010082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ec88-2313-42ee-b9d0-3afe9d30cdb4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.010082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 213/434] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:08.010987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ec88-2313-42ee-b9d0-3afe9d30cdb4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.076413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:08.097737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ceab6-d280-4085-b870-8a4764f5c16c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.138500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cb2ac-2064-4516-b28c-1d77014adf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.139421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cb2ac-2064-4516-b28c-1d77014adf58 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.139702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cb2ac-2064-4516-b28c-1d77014adf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.139990 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cb2ac-2064-4516-b28c-1d77014adf58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.149938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ec88-2313-42ee-b9d0-3afe9d30cdb4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.154743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cb2ac-2064-4516-b28c-1d77014adf58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-242cb2ac-2064-4516-b28c-1d77014adf58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:08.159710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:08.160639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:08.161460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 222/435] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:08.162700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23205f78-fcfa-4791-b1f9-309b29410f03 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.162700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23205f78-fcfa-4791-b1f9-309b29410f03 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.162700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23205f78-fcfa-4791-b1f9-309b29410f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.181354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-23205f78-fcfa-4791-b1f9-309b29410f03 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:08.182583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-23205f78-fcfa-4791-b1f9-309b29410f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:08.195269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.195904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.196184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.196459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.197616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:08.199071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:08.200864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 214/436] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:08.201470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.261734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 223/437] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:08.262616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23205f78-fcfa-4791-b1f9-309b29410f03 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.288416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:08.326153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5737-14f1-4848-92c0-64e68b790c17 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.344135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:08.363911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.364921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.365263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.365598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.366373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:08.366736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:08.368178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:08.376646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 215/438] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:08.376800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23205f78-fcfa-4791-b1f9-309b29410f03 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.377545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.416400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.417266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.418168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.418460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.419188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=3530f0bc598248418cc693afa8f4b150, scope.project.id=67493e851cd54135a5319695e18189a1) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:08.419498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:08.420916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:08.427757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 224/439] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=3530f0bc598248418cc693afa8f4b150 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:08.429356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.487800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde859e-749e-42e4-8016-579e16738990 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.526012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.526850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.527792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.528132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.539959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:08.546713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:08.547721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 216/440] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:08.548964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.562544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:08.576078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95ad3-d28e-4dd2-ae01-ce9edd436840 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.597595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:08.614936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.615568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.615911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.616194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.671024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad4b23-6159-4a1d-8797-0942f23f414d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.703122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:08.705359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:08.710534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.711158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.711435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.711701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.722135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:08.727002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:08.728343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 217/441] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:08.729226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.795059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:08.809747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 225/442] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:08.810609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.830336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:08.834499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143eaf3-406d-455a-816d-1f6c75118f7e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.867675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.868276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.868543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.868813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.869752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:08.870087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:08.871528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:08.879332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 218/443] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:08.880121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.909683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bc368-08b4-4886-aafb-5b8e1158a192 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:08.945020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:08.945621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:08.945935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:08.946211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:08.946880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:08.947186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:08.948543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:08.956591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 226/444] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:08.957388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:08.983875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1fafd-1faf-465e-bd5e-9ae22d6667d3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.006955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.017894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.018525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.018801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.019077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] PATH_INFO: `/v3/projects/eeb8255bb2f340b6aad60728a547a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.020189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb8255bb2f340b6aad60728a547a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.021616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.023425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 219/445] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/projects/eeb8255bb2f340b6aad60728a547a86c => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:09.024021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.052518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.062131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951fe1f-070e-4a73-b80e-9a4cf9d9c06f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.095640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.096236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.096516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.096790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.124008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227056f-afdb-4acc-872f-864bd17bcf15 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.160397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.161002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.161276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.161542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.169675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.171309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.230093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.232082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.257907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.269832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 227/446] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:09.270865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.330237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 220/447] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:09.331285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.374001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c49ef-e1fd-4978-afa4-0c7a3f4eaceb None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.408410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.409041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.409319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.409589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.410723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.412132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.413684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 228/448] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:09.414473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.434120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.436109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1a0ac-97cc-43ad-a30e-b6bd2f0f553a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.463349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.472467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.473099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.473379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.473659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] PATH_INFO: `/v3/projects/a73ceb365cbe45598f5feb33f3998086` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.474814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] RBAC: Authorizing `identity:get_project(project_id=a73ceb365cbe45598f5feb33f3998086)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.476237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.477923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 221/449] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/projects/a73ceb365cbe45598f5feb33f3998086 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:09.478682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.517177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192848d-b85c-4d91-a0b4-e9ad2ae7f14c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.560821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.561463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.561736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.562021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] PATH_INFO: `/v3/projects/67493e851cd54135a5319695e18189a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.563184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] RBAC: Authorizing `identity:get_project(project_id=67493e851cd54135a5319695e18189a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.564662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.566471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 229/450] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/projects/67493e851cd54135a5319695e18189a1 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:09.567332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.585691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4a65b-f023-4a66-9a66-d2b3b00710f0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.620115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.621245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.621538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.621874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.622694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=eeb8255bb2f340b6aad60728a547a86c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:09.622865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.624304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.630045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 222/451] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=eeb8255bb2f340b6aad60728a547a86c&role.id=0a108a4463a4458480b00aed843b4e9b => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:09.630817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.656884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.676690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195ab82-13ad-4054-b7aa-628615b84dec None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.713982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:09.718497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.719215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.719532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.719807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] PATH_INFO: `/v3/groups/c6f933d34324493294b2b8092c8e7c7f` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.720991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f933d34324493294b2b8092c8e7c7f)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.722774 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.729402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 230/452] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/groups/c6f933d34324493294b2b8092c8e7c7f => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:09.731080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.767138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464c5a-59ff-4a91-8283-16ce47aaf836 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.808714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.809569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.809783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.810079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] PATH_INFO: `/v3/groups/718045f856084708a32bc015147ad1e8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.811220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] RBAC: Authorizing `identity:get_group(group_id=718045f856084708a32bc015147ad1e8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.812639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.818012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 223/453] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/groups/718045f856084708a32bc015147ad1e8 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:09.820665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:09.852855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd925ff-06cc-4f3b-8122-25208c202197 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.896265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4c704-7755-4ee2-9c63-8006bec50e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:09.897018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4c704-7755-4ee2-9c63-8006bec50e06 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:09.897364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4c704-7755-4ee2-9c63-8006bec50e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:09.897706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4c704-7755-4ee2-9c63-8006bec50e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:09.898526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4c704-7755-4ee2-9c63-8006bec50e06 None admin] RBAC: Adding query filter params (group.id=c6f933d34324493294b2b8092c8e7c7f, role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=a73ceb365cbe45598f5feb33f3998086) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:09.898995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4c704-7755-4ee2-9c63-8006bec50e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:09.900462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4c704-7755-4ee2-9c63-8006bec50e06 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:09.911525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 231/454] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/role_assignments?group.id=c6f933d34324493294b2b8092c8e7c7f&scope.project.id=a73ceb365cbe45598f5feb33f3998086&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:09.958005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39325c5-0422-439e-b909-fc53eb3c7131 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:09.998346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:10.003294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6241b2c-8832-4669-ac53-4804ca3023a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:10.003923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6241b2c-8832-4669-ac53-4804ca3023a7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:10.004211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6241b2c-8832-4669-ac53-4804ca3023a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:10.004494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6241b2c-8832-4669-ac53-4804ca3023a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:10.005595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6241b2c-8832-4669-ac53-4804ca3023a7 None admin] RBAC: Adding query filter params (group.id=718045f856084708a32bc015147ad1e8, role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=67493e851cd54135a5319695e18189a1) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:10.005961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6241b2c-8832-4669-ac53-4804ca3023a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:10.007675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6241b2c-8832-4669-ac53-4804ca3023a7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:10.015626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 224/455] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Sep 19 11:44:09 2022] GET /identity/v3/role_assignments?group.id=718045f856084708a32bc015147ad1e8&scope.project.id=67493e851cd54135a5319695e18189a1&role.id=0a108a4463a4458480b00aed843b4e9b => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:12.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2a49-389c-465a-ad6d-795cf4bc85f2 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:12.872792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2a49-389c-465a-ad6d-795cf4bc85f2 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:12.872792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2a49-389c-465a-ad6d-795cf4bc85f2 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:12.874401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 232/456] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:12.891749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db697-62cd-4f78-a0d7-728832f7e99c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:12.892214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db697-62cd-4f78-a0d7-728832f7e99c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:12.892297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db697-62cd-4f78-a0d7-728832f7e99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:12.918781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b14db697-62cd-4f78-a0d7-728832f7e99c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:12.920004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b14db697-62cd-4f78-a0d7-728832f7e99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:13.014127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 225/457] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:13.128987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa3ed-9e65-485a-90d7-52bf06ab7f84 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.128987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa3ed-9e65-485a-90d7-52bf06ab7f84 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.129229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa3ed-9e65-485a-90d7-52bf06ab7f84 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.131363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 233/458] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:13.136981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d0a1-1971-40ea-9c70-e6e4d369696f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.137267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d0a1-1971-40ea-9c70-e6e4d369696f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.137540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d0a1-1971-40ea-9c70-e6e4d369696f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.162309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c594d0a1-1971-40ea-9c70-e6e4d369696f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:13.163529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c594d0a1-1971-40ea-9c70-e6e4d369696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:13.263599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 226/459] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:13.287108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451e8f8-660e-49b9-bfe0-9ad3fb2f5c9d None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.287414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451e8f8-660e-49b9-bfe0-9ad3fb2f5c9d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.287664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451e8f8-660e-49b9-bfe0-9ad3fb2f5c9d None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.289126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 234/460] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:13.295163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699c28-1501-4f87-a96f-d62bf3bb786b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.295450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699c28-1501-4f87-a96f-d62bf3bb786b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.295725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699c28-1501-4f87-a96f-d62bf3bb786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.322773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1c699c28-1501-4f87-a96f-d62bf3bb786b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:13.324490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c699c28-1501-4f87-a96f-d62bf3bb786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:13.366347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451e8f8-660e-49b9-bfe0-9ad3fb2f5c9d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:13.385920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:13.392539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:13.423977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 227/461] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:13.540394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451e8f8-660e-49b9-bfe0-9ad3fb2f5c9d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:13.592486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4635d-890a-4c60-bd3c-e62e63c5c76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:13.593237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4635d-890a-4c60-bd3c-e62e63c5c76c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.593513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4635d-890a-4c60-bd3c-e62e63c5c76c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.593801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4635d-890a-4c60-bd3c-e62e63c5c76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.611647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4635d-890a-4c60-bd3c-e62e63c5c76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:13.613094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4635d-890a-4c60-bd3c-e62e63c5c76c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-06e4635d-890a-4c60-bd3c-e62e63c5c76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:13.633547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:13.634558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:13.635469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:13.635469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:13.635469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:13.635469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 235/462] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:13.641440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c699c28-1501-4f87-a96f-d62bf3bb786b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:13.715162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd079fa-45f7-406d-aa6f-22f6d8eb0aa5 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.715162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd079fa-45f7-406d-aa6f-22f6d8eb0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.715162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd079fa-45f7-406d-aa6f-22f6d8eb0aa5 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.715162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 236/463] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:13.719997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd079fa-45f7-406d-aa6f-22f6d8eb0aa5 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:13.791112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c699c28-1501-4f87-a96f-d62bf3bb786b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:13.798492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:13.833577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:13.834207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.834483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.834749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.835379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:13.835675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:13.836213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd079fa-45f7-406d-aa6f-22f6d8eb0aa5 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:13.837028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:13.849257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 228/464] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:13.849944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:13.876814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f851dc3-b0cf-4819-af06-cbf887ff1c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:13.877443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f851dc3-b0cf-4819-af06-cbf887ff1c87 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.877712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f851dc3-b0cf-4819-af06-cbf887ff1c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.878007 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f851dc3-b0cf-4819-af06-cbf887ff1c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.890234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f851dc3-b0cf-4819-af06-cbf887ff1c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:13.891979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f851dc3-b0cf-4819-af06-cbf887ff1c87 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:13.899796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-6f851dc3-b0cf-4819-af06-cbf887ff1c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:13.908779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:13.909738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:13.910762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:13.910762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:13.910762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:13.912105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 237/465] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:13.913261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea99124-baaf-4ce5-a438-2ceab9c6e525 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.913548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea99124-baaf-4ce5-a438-2ceab9c6e525 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.913863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea99124-baaf-4ce5-a438-2ceab9c6e525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.934779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-cea99124-baaf-4ce5-a438-2ceab9c6e525 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:13.935973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cea99124-baaf-4ce5-a438-2ceab9c6e525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:13.961511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28880899-694f-471d-9939-1c1a2374b7fb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:13.994518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:13.995144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.995424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.996325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.006870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.008309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.011551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 238/466] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:14.012201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea99124-baaf-4ce5-a438-2ceab9c6e525 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:14.028549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:14.029527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:14.030416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:14.030416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:14.030416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:14.030416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 229/467] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:14.030416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:14.076622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:14.138203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:14.146505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495658cd-ee24-44be-acad-f4dbf3e0a96e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:14.157693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea99124-baaf-4ce5-a438-2ceab9c6e525 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:14.185725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:14.186690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.187108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.187519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.188405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:14.188961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.190693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.206009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:14.206271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 230/468] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:14.206596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.206876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.207151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.207816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.208013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:14.209409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.238967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 239/469] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:44:14 2022] POST /identity/v3/users => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:14.240226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:14.332614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdac64-1307-4006-86f0-0b9fed240c8f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:14.379743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:14.380365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.380643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.380920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.381572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:14.381918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.383323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.391470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e25f1-68ba-4067-8fee-ce83b6b80724 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:14.397976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 231/470] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:14.398559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:14.406653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:14.418303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:14.438830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10f45a-0ce1-44b0-b319-b0b76da68b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:14.439452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10f45a-0ce1-44b0-b319-b0b76da68b5e None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.439730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10f45a-0ce1-44b0-b319-b0b76da68b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.439996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10f45a-0ce1-44b0-b319-b0b76da68b5e None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.440832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10f45a-0ce1-44b0-b319-b0b76da68b5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.444561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10f45a-0ce1-44b0-b319-b0b76da68b5e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.470338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 240/471] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:44:14 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:14.471420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9fba9-393c-4c21-9c7f-6af52ea90845 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.471704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9fba9-393c-4c21-9c7f-6af52ea90845 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.471976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9fba9-393c-4c21-9c7f-6af52ea90845 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.473460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 241/472] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:14.474245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9fba9-393c-4c21-9c7f-6af52ea90845 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:14.527950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95925a52-05d9-4eed-9f53-3b3c3a8e73ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:14.588182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2dfc2-2436-40ee-8fd9-f4ab4601a550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:14.590547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2dfc2-2436-40ee-8fd9-f4ab4601a550 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.591044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2dfc2-2436-40ee-8fd9-f4ab4601a550 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.591541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2dfc2-2436-40ee-8fd9-f4ab4601a550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.609723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2dfc2-2436-40ee-8fd9-f4ab4601a550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.612328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2dfc2-2436-40ee-8fd9-f4ab4601a550 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.623507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9fba9-393c-4c21-9c7f-6af52ea90845 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0fb2dfc2-2436-40ee-8fd9-f4ab4601a550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:14.629840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:14.630942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:14.631813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:14.631813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:14.631813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:14.631813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 232/473] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:14.632649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6c07f-f203-4949-9943-8eeb46f6d63f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.638233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6c07f-f203-4949-9943-8eeb46f6d63f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.638553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6c07f-f203-4949-9943-8eeb46f6d63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.661186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-38c6c07f-f203-4949-9943-8eeb46f6d63f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:14.663349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-38c6c07f-f203-4949-9943-8eeb46f6d63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:14.671375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:14.672404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.672886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.673392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.675773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.676108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.703104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 242/474] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:44:14 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:14.704602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:14.740774 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:14.778226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:14.790685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 233/475] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:14.860235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590a433-0da7-4afb-b7e2-96e5725e77bc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:14.900254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:14.900878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:14.901187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:14.901545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:14.902855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:14.903245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:14.904718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:14.919483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 243/476] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:14.924794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6c07f-f203-4949-9943-8eeb46f6d63f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:15.048796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6c07f-f203-4949-9943-8eeb46f6d63f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:15.094532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:15.095145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:15.095417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:15.095882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:15.096809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:15.098266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:15.123544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 234/477] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:44:14 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:15.155388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:15.262216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:15.287744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8e79f-91fc-44f5-974b-b302e9651cc7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:15.328400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:15.329646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:15.330143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:15.330623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:15.331506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:15.333088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:15.342515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.sql.core [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:44:15.342515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 11:44:15.342515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [parameters: {'id': 'db02966c688749db8c2564d2c914cfe7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 11:44:15.342515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81405) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application result.read() Sep 19 11:44:15.360684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return ref.to_dict() Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:44:15.361649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application yield resource Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise self.value Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application session.commit() Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:44:15.362599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:44:15.363466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:44:15.364411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application result.read() Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:44:15.365262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [parameters: {'id': 'db02966c688749db8c2564d2c914cfe7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:15.367528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 11:44:15.369505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:15.373512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 244/478] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Sep 19 11:44:15 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:15.379227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:15.475339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:15.535299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c735-ff79-4d9d-bd4c-dacdf2b99482 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:15.583415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9c271-abe6-4db9-974e-1a718062dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:15.584061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c271-abe6-4db9-974e-1a718062dd93 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:15.584342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c271-abe6-4db9-974e-1a718062dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:15.584615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c271-abe6-4db9-974e-1a718062dd93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:15.598054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9c271-abe6-4db9-974e-1a718062dd93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9cd9c271-abe6-4db9-974e-1a718062dd93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:15.603324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:15.606054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:15.607994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 235/479] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:15.613159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:15.747670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9af46f-86b5-4a54-8e69-3cdf78cfe114 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:15.786634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218534e2-7335-4129-9b7f-ae90803692f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:15.787596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218534e2-7335-4129-9b7f-ae90803692f4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:15.788075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218534e2-7335-4129-9b7f-ae90803692f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:15.788541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218534e2-7335-4129-9b7f-ae90803692f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:15.789421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218534e2-7335-4129-9b7f-ae90803692f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:15.789948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218534e2-7335-4129-9b7f-ae90803692f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:15.791533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218534e2-7335-4129-9b7f-ae90803692f4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:15.800897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 245/480] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:16.973694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c467cd9-4355-4ab6-a4cf-0e58890d7e7b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:16.974276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c467cd9-4355-4ab6-a4cf-0e58890d7e7b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:16.974276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c467cd9-4355-4ab6-a4cf-0e58890d7e7b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:16.975849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 236/481] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:16.981896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183aecc-af5b-4d72-9502-483695fabef6 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:16.982289 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183aecc-af5b-4d72-9502-483695fabef6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:16.982452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183aecc-af5b-4d72-9502-483695fabef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:16.998995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:17.010656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3183aecc-af5b-4d72-9502-483695fabef6 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:17.013234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3183aecc-af5b-4d72-9502-483695fabef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:17.094591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 246/482] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:17.380175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc69a-df0f-498a-9b63-d26042136bca None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:17.380570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc69a-df0f-498a-9b63-d26042136bca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:17.380741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc69a-df0f-498a-9b63-d26042136bca None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:17.382351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 237/483] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:17.387909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4338d0-59d1-472d-8874-f29d1bfd141b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:17.388191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4338d0-59d1-472d-8874-f29d1bfd141b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:17.388460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4338d0-59d1-472d-8874-f29d1bfd141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:17.436923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fe4338d0-59d1-472d-8874-f29d1bfd141b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:17.438163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fe4338d0-59d1-472d-8874-f29d1bfd141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:17.547061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc69a-df0f-498a-9b63-d26042136bca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:17.569345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 247/484] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:17.780800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc69a-df0f-498a-9b63-d26042136bca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:17.801697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b238c5-c168-4fe6-babd-d24bcae84064 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:17.802252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b238c5-c168-4fe6-babd-d24bcae84064 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:17.802663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b238c5-c168-4fe6-babd-d24bcae84064 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:17.804178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 248/485] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:17.808860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:17.819412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25c3ae-3704-45b4-bd4a-19d93fd72ca5 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:17.820004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25c3ae-3704-45b4-bd4a-19d93fd72ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:17.820503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25c3ae-3704-45b4-bd4a-19d93fd72ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:17.836972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:17.837860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:17.838373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:17.838836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:17.851626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:17.857353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:17.858385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:17.859216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:17.862057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 238/486] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:17.870666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:17.882511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-6d25c3ae-3704-45b4-bd4a-19d93fd72ca5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:17.884045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6d25c3ae-3704-45b4-bd4a-19d93fd72ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:18.026139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa851b90-2888-4b4e-880c-a1ede4d51180 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.035946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.044642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 249/487] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:18.045705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25c3ae-3704-45b4-bd4a-19d93fd72ca5 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.071605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c3861-a99c-44b1-bb92-af5d9d616d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.072601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c3861-a99c-44b1-bb92-af5d9d616d0c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.073075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c3861-a99c-44b1-bb92-af5d9d616d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.074249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c3861-a99c-44b1-bb92-af5d9d616d0c None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.075090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c3861-a99c-44b1-bb92-af5d9d616d0c None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:18.075648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c3861-a99c-44b1-bb92-af5d9d616d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:18.082093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c3861-a99c-44b1-bb92-af5d9d616d0c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:18.094849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 239/488] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:18.096613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33d66-9517-4478-960e-1e5303470bdd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.097079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33d66-9517-4478-960e-1e5303470bdd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.097747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33d66-9517-4478-960e-1e5303470bdd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.099472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 240/489] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:18.100521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d33d66-9517-4478-960e-1e5303470bdd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.155385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.179756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25c3ae-3704-45b4-bd4a-19d93fd72ca5 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.219478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31758fb6-b328-4cb9-bfb2-cc1f840d5d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.220572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758fb6-b328-4cb9-bfb2-cc1f840d5d93 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.221140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758fb6-b328-4cb9-bfb2-cc1f840d5d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.221617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758fb6-b328-4cb9-bfb2-cc1f840d5d93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.230324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d33d66-9517-4478-960e-1e5303470bdd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.234065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31758fb6-b328-4cb9-bfb2-cc1f840d5d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-31758fb6-b328-4cb9-bfb2-cc1f840d5d93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:18.239589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:18.240527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:18.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:18.249253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 250/490] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:18.253033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973da28-08f9-455a-9606-3d4878898a46 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.253660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973da28-08f9-455a-9606-3d4878898a46 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.254203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973da28-08f9-455a-9606-3d4878898a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.269207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.269890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.270179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.270441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.276214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0973da28-08f9-455a-9606-3d4878898a46 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:18.277935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0973da28-08f9-455a-9606-3d4878898a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:18.282983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:18.284433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:18.300442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:18.301405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:18.302391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:18.302391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:18.302391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:18.302997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 241/491] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:18.308591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.338573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.357928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 251/492] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:18.359063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973da28-08f9-455a-9606-3d4878898a46 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.418347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.438043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce16e59-6b03-4ad6-a0fe-71dd13320bd4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.483317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.484290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.484575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.484914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.485904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:18.486214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:18.488066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:18.494162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973da28-08f9-455a-9606-3d4878898a46 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.504185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 242/493] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:18.504185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.554824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.556206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.556673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.557395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.558332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:18.559107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:18.561529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:18.578877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 252/494] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:18.581042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.630122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23228b-a8fc-483d-9530-cb8dcaff946d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.635098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.672446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.673071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.673352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.673628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.684348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:18.689702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:18.690714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:18.691531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 243/495] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:18.692208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c4644-d912-4904-a818-94f3e657092f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.692279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.706395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.727412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.728028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.728301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.728570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.738972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:18.740411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:18.754575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:18.755833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:18.756711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:18.756711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:18.756711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:18.756711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 253/496] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:18.756711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.799013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c157538-e1bf-41d6-a392-80f9b8b3ff73 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.832767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5ab90-1260-4726-990b-ff5bdb427fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.833302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ab90-1260-4726-990b-ff5bdb427fab None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.833586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ab90-1260-4726-990b-ff5bdb427fab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.833890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ab90-1260-4726-990b-ff5bdb427fab None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.841341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5ab90-1260-4726-990b-ff5bdb427fab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:18.842849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5ab90-1260-4726-990b-ff5bdb427fab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:18.851533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.855597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9fb4-f92c-4fe1-85fc-a8fdb242fd03 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fcc5ab90-1260-4726-990b-ff5bdb427fab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:18.856005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:18.856882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:18.857822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 244/497] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:18.858982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43791e4f-b4b3-4533-81a8-964972a13bbc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.859080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43791e4f-b4b3-4533-81a8-964972a13bbc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.859281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43791e4f-b4b3-4533-81a8-964972a13bbc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.865191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 245/498] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:18.865191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43791e4f-b4b3-4533-81a8-964972a13bbc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.893013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:18.893634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:18.893935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:18.894199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:18.905316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:18.910488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:18.911506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 254/499] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:18.912336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:18.934405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:18.972545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43791e4f-b4b3-4533-81a8-964972a13bbc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.005372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.005999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.006275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.006543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.007196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:19.007497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.008864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.015971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed0049-df86-4f14-bca1-7b0d7cfebab9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.016612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 246/500] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:19.017384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.053849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dccfa3-0329-4300-a749-89bc19cb16a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.054452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dccfa3-0329-4300-a749-89bc19cb16a7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.054722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dccfa3-0329-4300-a749-89bc19cb16a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.054994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dccfa3-0329-4300-a749-89bc19cb16a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.055631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dccfa3-0329-4300-a749-89bc19cb16a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:19.055934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dccfa3-0329-4300-a749-89bc19cb16a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.057292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dccfa3-0329-4300-a749-89bc19cb16a7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.068665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 255/501] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:19.069865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057a11-8ec3-4cf9-9ea8-8945e9ac8a13 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.070149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057a11-8ec3-4cf9-9ea8-8945e9ac8a13 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.070418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057a11-8ec3-4cf9-9ea8-8945e9ac8a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.088633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2d057a11-8ec3-4cf9-9ea8-8945e9ac8a13 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:19.090326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2d057a11-8ec3-4cf9-9ea8-8945e9ac8a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:19.103996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.126487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65a7c4-f2ae-422c-8e63-3c95b0c05b98 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.157119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.167384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 256/502] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:19.167384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d057a11-8ec3-4cf9-9ea8-8945e9ac8a13 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.170328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.170934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.171214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.171511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.172241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:19.172552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.174012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.211513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 247/503] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:19.212346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.273579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d057a11-8ec3-4cf9-9ea8-8945e9ac8a13 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.312437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.313035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.313316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.313584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.314276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:19.314574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.315926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.322736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48251f-2bf7-4f5f-8841-8afeb56167c6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.326137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 257/504] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:19.326392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.345173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.359906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.360541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.360821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.361118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.373305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.374797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:19.390618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:19.391562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:19.392453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:19.392453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:19.392453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:19.392453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 248/505] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:19.393129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.436612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.447767 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c33db-0835-4a66-8125-bb42cb3a2fef None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.489525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.490190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.490467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.490744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.499110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.499242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e88d7-2499-415f-85ac-36b71536e18f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.500543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:19.514491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:19.515592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 258/506] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:19.516757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.537971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.538571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.538843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.539110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.586213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.606183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.607606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.616952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033012c-a103-4f50-8250-60b79bdad9e0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.649494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.653228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.653890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.654263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.654494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.665058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.666645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:19.680590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:19.681742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:19.683313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:19.683313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:19.683313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:19.683313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 259/507] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:19.683313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.712195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 249/508] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:19.713038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.787332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef9f5-94df-406f-9eaf-13a7f2a88cb2 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.811363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.825949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.826227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccb130-17cb-4620-87fd-05b607b9d2ed None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.826775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.827134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.827533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.828319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:19.828971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.830455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.842802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 260/509] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:19.843628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.861709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.862297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.862576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.862854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.871426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:19.873690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.875714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:19.890972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:19.891932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:19.892802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:19.892802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:19.892802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:19.892802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 250/510] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:19.892802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:19.951004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db157665-06a1-47e8-937c-90c27440d563 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:19.986260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:19.986852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:19.987125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:19.987388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:19.988108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:19.988412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:19.989849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:19.996498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb712c7-b74d-4bca-9273-b26f93398e82 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.025421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 261/511] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:20.026291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.034656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.035271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.035608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.035887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.036536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:20.036841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.038254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.050673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 251/512] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:20.051507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.073033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.116436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.144419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c12808-21f4-460a-9aea-987e65b225dd None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.184686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.185333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.185606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.185904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.186823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db218523-877a-4a3d-a521-6c9703edf7fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.196426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.197915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:20.214467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:20.215607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:20.216703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:20.216703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:20.216703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:20.216703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 262/513] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:20.216703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.226665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.227291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.227565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.227836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.235849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.237284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:20.248694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:20.249683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 252/514] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:20.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.314316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.324673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cc8d7-0857-4266-92ac-0c7a620ccf30 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.352409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56823cbd-796e-41fc-91d0-e8c2bdc664ab None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.361576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.362213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.362498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.362772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.363424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:20.363729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.365093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.366027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.377429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 263/515] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:20.378338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.393069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.393672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.393978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.394245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.466374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.467850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.494496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bf513-e619-4ac8-882e-0e705f2fa6ae None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.528782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.529420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.529697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.530002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.537572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.539070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:20.551036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:20.552176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 264/516] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:20.553116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.568317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.579531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 253/517] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:20.580388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.598399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.651841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6034c-ae43-4e93-9a22-f20958e89a8d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.684407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4cd49-2921-4e1b-864b-bf79fbd16f91 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.685404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.686033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.686319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.686727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.687238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:20.687542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.688902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.700434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 265/518] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:20.701220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.718914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.719526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.719805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.720078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.720806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:20.721114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.722581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.740009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 254/519] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:20.740823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.771435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.811513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:20.819683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f646-d1ba-4c1c-a55f-235c0b35eace None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.843013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac9fd-2fe6-450a-a7b0-b743472837fd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:20.854366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.855456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.855731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.855999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.856645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.858099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.878177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:20.878802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:20.879074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:20.879345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:20.879546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 266/520] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:44:20 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:20.880096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:20.880481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:20.881801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:20.882714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:20.911964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 255/521] 10.222.0.88 () {56 vars in 1139 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:20.912893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.006505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c782c-a0d5-4cd6-9607-f7325e65e860 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.038285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.047193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9fb5-9408-4291-a5f4-4c7e9d8ed8cb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.050182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.050835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.051119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.051391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.063355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.065008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.081026 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:21.083058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:21.084445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:21.085370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:21.085370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:21.085370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:21.085370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 267/522] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:21.086558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.093199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.094600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.094600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.094600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.102029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.103760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:21.117304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:21.118470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:21.119422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:21.122030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 256/523] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:21.122960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.200129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3268251-e92e-4d41-b5d8-5fb688508d01 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.235245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.235847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.236124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.236395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.239665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e96f38-63df-4c06-a577-689230c75468 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.275893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.276550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.276867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.277138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.291556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.316474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.317953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.336889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.350670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.352147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.432643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 268/524] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:21.434239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.469706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 257/525] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:21.470406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.523309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.546311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa638459-6e60-422e-8918-a6feefbdd169 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.564543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.579529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c57ec-eecc-4867-881e-c1d2d527312b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.583076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.583685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.583967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.584239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.584888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:21.585195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.586586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.603242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 269/526] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:21.603242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.619731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.620399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.620674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.620942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.621627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:21.621960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.623370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.646952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 258/527] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:21.646952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.718637 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393d545-680c-4724-b3aa-76350806c01d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.755818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.756420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.756537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f930132-921b-496d-a8a1-04d64a06bc51 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.756701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.756950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.766927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.769680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.771301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:21.787556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:21.788596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:21.789496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:21.789496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:21.789496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:21.793073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.793965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 270/528] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:21.794070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.794299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.794567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.794845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.809220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.811071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.820930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:21.829000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:21.835188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:21.836286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:21.836286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:21.836286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:21.836286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 259/529] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:21.836286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.907684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b899918-c90f-4479-b891-04aef2234812 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.937563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d9ca1-cb35-4db2-a306-de766926e8fd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:21.947860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.948849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.949135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.949409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.957535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.959104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.971282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:21.971872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:21.972147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:21.972412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:21.972693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:21.973681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:21.974815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:21.975333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=345c823f8d85402ba3b07f39ed5e2fca) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:21.975333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:21.975333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:21.976621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 271/530] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:21.986144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 260/531] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/role_assignments?user.id=345c823f8d85402ba3b07f39ed5e2fca&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:21.986144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:21.986409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.065313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:22.087009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:22.126442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224000bd-b0d3-44a1-941d-27b8e1adf0d4 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.148921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0aec2-5147-45e0-a99d-68772cdded9a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.173173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.174018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.174420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.174757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.176291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:22.176667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.178230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.193740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 272/532] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:22.194751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.195149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.195917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.196354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.196765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.197589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:22.198371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.200898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.217547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 261/533] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:22.218814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.312533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e2eb-bee0-42b5-bf11-bbfd63b2b230 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.331757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:22.350529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdea09-ed08-4fd3-be0e-e619b364136f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.364277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.365418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.365889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.366319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.367470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:22.368495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.370954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.379277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:22.392691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.393437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.393843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.394242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.402206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 273/534] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:22.402206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.407249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.408788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:22.421267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:22.422382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:22.423457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:22.424911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 262/535] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:22.425873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.531979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7baad8-f05e-46dd-9e78-7727d3a327e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.542632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478ee9f-2f77-49b1-9b6c-d4995ea2db18 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.568043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.568808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.569180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.570036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.570892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=37099ac7c39948f0afd272a2b41949ee) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:22.571445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.573434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.585876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 263/536] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/role_assignments?user.id=37099ac7c39948f0afd272a2b41949ee&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:22.585876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.597628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.598309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.598587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.598855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.611662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.613131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:22.640059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:22.640985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:22.641879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:22.642500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 274/537] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:22.652283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.699902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:22.716083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:22.787795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe14c0-c03a-44a5-a545-7e7aadf6b43b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.819107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87cb4-1cfb-4cdb-804b-058ebdc738c2 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:22.837461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.838479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.839071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.839544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.840583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:22.841205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.843153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.859017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 264/538] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:22.862717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:22.862717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:22.862717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:22.862717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:22.863059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:22.863059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:22.863507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:22.866513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.879352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 275/539] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:22.886520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:22.994282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:23.006543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a3d24-7ce3-4584-8b7e-23c592794097 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:23.070903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688418d-7aac-4d1c-8e87-5af8fd7450fb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:23.086806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:23.110140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4e89c-689c-49fd-ab21-e7ad187cfc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:23.110767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e89c-689c-49fd-ab21-e7ad187cfc60 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:23.111048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e89c-689c-49fd-ab21-e7ad187cfc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:23.111315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e89c-689c-49fd-ab21-e7ad187cfc60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:23.112039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4e89c-689c-49fd-ab21-e7ad187cfc60 None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=6a24bc4a0c4f480192d87a27a243acb1) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:23.112349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4e89c-689c-49fd-ab21-e7ad187cfc60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:23.113791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4e89c-689c-49fd-ab21-e7ad187cfc60 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:23.119377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 265/540] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/role_assignments?user.id=6a24bc4a0c4f480192d87a27a243acb1&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:23.123799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb9395-78c0-493a-bd76-e285caab9f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:23.124581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb9395-78c0-493a-bd76-e285caab9f2a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:23.124687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb9395-78c0-493a-bd76-e285caab9f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:23.124941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb9395-78c0-493a-bd76-e285caab9f2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:23.125676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb9395-78c0-493a-bd76-e285caab9f2a None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=dee035c97cb6447382365faf6b6aca53) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:23.126050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb9395-78c0-493a-bd76-e285caab9f2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:23.127502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb9395-78c0-493a-bd76-e285caab9f2a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:23.134208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 276/541] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:22 2022] GET /identity/v3/role_assignments?user.id=dee035c97cb6447382365faf6b6aca53&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:23.189628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836789-0ef6-4801-829d-44dcc5109f93 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:23.189920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836789-0ef6-4801-829d-44dcc5109f93 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:23.190200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836789-0ef6-4801-829d-44dcc5109f93 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:23.191712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 266/542] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:23.197647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1f815-1fe4-4bf6-9875-6bf80a3a7d20 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:23.198197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1f815-1fe4-4bf6-9875-6bf80a3a7d20 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:23.198664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1f815-1fe4-4bf6-9875-6bf80a3a7d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:23.218988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4ca1f815-1fe4-4bf6-9875-6bf80a3a7d20 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:23.220473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4ca1f815-1fe4-4bf6-9875-6bf80a3a7d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:23.295648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 277/543] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:23.569829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836789-0ef6-4801-829d-44dcc5109f93 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:23.707939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836789-0ef6-4801-829d-44dcc5109f93 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:23.757304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:23.758297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:23.758759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:23.759213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:23.778866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:23.784179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:23.785074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:23.785939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 267/544] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:23.791505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1f815-1fe4-4bf6-9875-6bf80a3a7d20 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:23.899799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:23.920473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1f815-1fe4-4bf6-9875-6bf80a3a7d20 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:23.960410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:23.961018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:23.961293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:23.961568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:23.962305 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:23.962611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:23.963996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:23.972612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 278/545] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:23.978247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:24.031182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:24.113335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507408-9a12-4f27-b058-f299109231c7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:24.169447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:24.178345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:24.178345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:24.178345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:24.185343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:24.186919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:24.212198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:24.213185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:24.214221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:24.214221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:24.214221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:24.214221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 268/546] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:24.219152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:24.371423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5395ad-e6a1-44a3-b727-407ba699df1d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:24.408096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310a893-e203-48ac-b265-8bb6fc70e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:24.408730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310a893-e203-48ac-b265-8bb6fc70e0ea None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:24.409010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310a893-e203-48ac-b265-8bb6fc70e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:24.409286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310a893-e203-48ac-b265-8bb6fc70e0ea None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:24.409988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310a893-e203-48ac-b265-8bb6fc70e0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:24.410296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310a893-e203-48ac-b265-8bb6fc70e0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:24.411720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310a893-e203-48ac-b265-8bb6fc70e0ea None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:24.426480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 279/547] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:24.434855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:24.493170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203e297-0257-440e-8354-dd3cf2785ed6 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:24.493474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203e297-0257-440e-8354-dd3cf2785ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:24.493715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203e297-0257-440e-8354-dd3cf2785ed6 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:24.495385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 280/548] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:24.501842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9da2b-b88b-4398-9399-5f8fef23fc84 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:24.502143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9da2b-b88b-4398-9399-5f8fef23fc84 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:24.502385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9da2b-b88b-4398-9399-5f8fef23fc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:24.523473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-54c9da2b-b88b-4398-9399-5f8fef23fc84 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:24.524690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-54c9da2b-b88b-4398-9399-5f8fef23fc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:24.526940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:24.569999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:24.600007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fae67-592d-41bb-b08c-1d8057da2f0c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:24.634573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 281/549] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:24.647593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:24.648217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:24.648492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:24.649006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:24.657099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:24.658576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:24.670098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:24.671243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:24.672184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 269/550] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:24.677159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9da2b-b88b-4398-9399-5f8fef23fc84 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:24.794860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9da2b-b88b-4398-9399-5f8fef23fc84 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:24.841363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:24.842000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:24.842334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:24.842614 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:24.843384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:24.843707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:24.845130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:24.858117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:24.868063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 282/551] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Sep 19 11:44:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:24.873863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.000050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a0e4-84c0-40cb-9796-3b1f68d0aed1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:25.008522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.047991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:25.049093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.049370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.049639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.082177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:25.179933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:25.181393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:25.190162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fad5f5-debe-411a-8dea-fd1d7f6b6800 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:25.252630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:25.253236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.253507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.261795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.284241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:25.299794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:25.300845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:25.301702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 283/552] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:25.308810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.391376 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 270/553] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:25.398453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.409266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:25.449793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:25.458190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18953-f07c-4811-a1f7-4be3a2f581f2 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:25.496414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26507a42-81a2-4437-82f6-085b1f00b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:25.496708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26507a42-81a2-4437-82f6-085b1f00b737 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.496994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26507a42-81a2-4437-82f6-085b1f00b737 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.497267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26507a42-81a2-4437-82f6-085b1f00b737 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.498032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26507a42-81a2-4437-82f6-085b1f00b737 None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:25.498590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26507a42-81a2-4437-82f6-085b1f00b737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:25.500033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26507a42-81a2-4437-82f6-085b1f00b737 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:25.509676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 284/554] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:25.510465 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d99462-445b-424e-b79e-7cee0246cf9f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:25.511507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082378f-66a9-48ae-bdc0-039da5bbd33e None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.511791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082378f-66a9-48ae-bdc0-039da5bbd33e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.512066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082378f-66a9-48ae-bdc0-039da5bbd33e None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.513586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 285/555] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:25.522794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082378f-66a9-48ae-bdc0-039da5bbd33e None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.550539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b7716-9c45-4905-acfe-fae360016d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:25.552008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7716-9c45-4905-acfe-fae360016d02 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.552428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7716-9c45-4905-acfe-fae360016d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.552842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7716-9c45-4905-acfe-fae360016d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.569428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b7716-9c45-4905-acfe-fae360016d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:25.572003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b7716-9c45-4905-acfe-fae360016d02 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e60b7716-9c45-4905-acfe-fae360016d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:25.596395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:25.597584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:25.598776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:25.598776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:25.598776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:25.598776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 271/556] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:25.600238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24dd1-650d-48d8-8462-ae15c51f2bb6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.600652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24dd1-650d-48d8-8462-ae15c51f2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.601084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24dd1-650d-48d8-8462-ae15c51f2bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.626885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-25d24dd1-650d-48d8-8462-ae15c51f2bb6 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:25.628252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-25d24dd1-650d-48d8-8462-ae15c51f2bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:25.647177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082378f-66a9-48ae-bdc0-039da5bbd33e None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:25.673666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:25.697865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:25.700014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.700516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.700983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.717613 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:25.725803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 272/557] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:25.725803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d24dd1-650d-48d8-8462-ae15c51f2bb6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.726352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:25.754847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:25.756337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:25.757255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:25.757255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:25.757255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:25.757255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 286/558] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:25.767649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.778430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:25.858909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d24dd1-650d-48d8-8462-ae15c51f2bb6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:25.895350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199427dd-60df-41e4-a7c3-178e83cc28f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:25.896329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199427dd-60df-41e4-a7c3-178e83cc28f0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.896831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199427dd-60df-41e4-a7c3-178e83cc28f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.897326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199427dd-60df-41e4-a7c3-178e83cc28f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.898193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199427dd-60df-41e4-a7c3-178e83cc28f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:25.898695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199427dd-60df-41e4-a7c3-178e83cc28f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:25.900273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199427dd-60df-41e4-a7c3-178e83cc28f0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:25.920052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 273/559] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:25.921864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c690-28fb-4e9a-87fc-869f18b82b67 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.922385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c690-28fb-4e9a-87fc-869f18b82b67 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.922851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c690-28fb-4e9a-87fc-869f18b82b67 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.924469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 274/560] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:25.925444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535c690-28fb-4e9a-87fc-869f18b82b67 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:25.939936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874588-fabf-4b6c-b282-51c4c8927590 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:25.975813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5edc0-179e-4416-8171-26788fc743a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:25.976418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5edc0-179e-4416-8171-26788fc743a7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.976695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5edc0-179e-4416-8171-26788fc743a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.976976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5edc0-179e-4416-8171-26788fc743a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:25.977602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5edc0-179e-4416-8171-26788fc743a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:25.977966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5edc0-179e-4416-8171-26788fc743a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:25.979976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5edc0-179e-4416-8171-26788fc743a7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:25.992831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 287/561] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:25.999290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c1163-1db1-4cf7-ba0d-9c2d48731ef8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:25.999603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c1163-1db1-4cf7-ba0d-9c2d48731ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:25.999840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c1163-1db1-4cf7-ba0d-9c2d48731ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.019511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-576c1163-1db1-4cf7-ba0d-9c2d48731ef8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:26.020722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-576c1163-1db1-4cf7-ba0d-9c2d48731ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:26.025857 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.044543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535c690-28fb-4e9a-87fc-869f18b82b67 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.048594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.085940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.086951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.087426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.087963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.093242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 288/562] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:26.094131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c1163-1db1-4cf7-ba0d-9c2d48731ef8 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.096154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.098085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:26.111237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:26.112189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:26.113084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:26.115500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 275/563] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:26.116701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.197884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c1163-1db1-4cf7-ba0d-9c2d48731ef8 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.231666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed297f9-9ad0-428e-990c-79c623d5515d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.236993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.237625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.237936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.238212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.246441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.248243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:26.260216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:26.267182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 289/564] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:26.268169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.273494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad23c5-bb8f-4fda-8953-29b0fe35ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.274137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad23c5-bb8f-4fda-8953-29b0fe35ac35 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.274724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad23c5-bb8f-4fda-8953-29b0fe35ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.274724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad23c5-bb8f-4fda-8953-29b0fe35ac35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.284830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad23c5-bb8f-4fda-8953-29b0fe35ac35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:26.288675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dbad23c5-bb8f-4fda-8953-29b0fe35ac35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:26.290063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:26.291125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:26.292025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 276/565] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:26.293226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd23ef-f75e-42eb-aa23-a525c6a04b4f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.293226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd23ef-f75e-42eb-aa23-a525c6a04b4f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.293226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd23ef-f75e-42eb-aa23-a525c6a04b4f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.294491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 277/566] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:26.295080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd23ef-f75e-42eb-aa23-a525c6a04b4f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.304159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.384174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3415d60-1275-4b01-a36e-ee03b697c0e2 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.405889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd23ef-f75e-42eb-aa23-a525c6a04b4f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.418207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.418816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.419099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.419381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.420075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:26.420386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.421836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.434163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 290/567] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:26.434964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.439461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28623a8-a241-4773-b607-6b1fa53efdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.440137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28623a8-a241-4773-b607-6b1fa53efdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.440420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28623a8-a241-4773-b607-6b1fa53efdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.440687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28623a8-a241-4773-b607-6b1fa53efdf6 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.441420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28623a8-a241-4773-b607-6b1fa53efdf6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:26.441724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28623a8-a241-4773-b607-6b1fa53efdf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.443176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28623a8-a241-4773-b607-6b1fa53efdf6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.465752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 278/568] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:26.465752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecff102-adc3-4f1a-96a6-3d893d2dcae1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.465752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecff102-adc3-4f1a-96a6-3d893d2dcae1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.465752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecff102-adc3-4f1a-96a6-3d893d2dcae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.480705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6ecff102-adc3-4f1a-96a6-3d893d2dcae1 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:26.481931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6ecff102-adc3-4f1a-96a6-3d893d2dcae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:26.522735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.545407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a264e18-b81e-47f0-beb4-abd34457e6f7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.546269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.555355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 279/569] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:26.556159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecff102-adc3-4f1a-96a6-3d893d2dcae1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.581072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.581676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.581990 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.582264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.582926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:26.583230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.584574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.592993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 291/570] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:26.594032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.666854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecff102-adc3-4f1a-96a6-3d893d2dcae1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.698986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a714d5-2adb-464a-b40f-ee2c526ac6fb None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.705136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.705814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.706261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.706740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.719422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:26.724434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:26.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 280/571] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:26.726414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.739988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.740596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.740877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.741150 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.742027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=1924693bdf934b15b1a784abfefdea5f) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:26.742391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.743957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.753932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 292/572] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/role_assignments?user.id=1924693bdf934b15b1a784abfefdea5f&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:26.755242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.769004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.822248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:26.849960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f524fb-2737-435a-98c5-792e3c205dec None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.888698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.889635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.889944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.890250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.897799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd2f27-36bb-45d5-93fb-d51855348060 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:26.902010 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.903518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:26.921598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:26.922832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:26.923779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:26.923779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:26.923779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:26.923779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 281/573] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:26.924735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:26.956980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:26.957723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:26.958059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:26.958303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:26.974144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:26.975891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:26.992031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:26.993365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:26.994809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:26.994809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:26.994809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:26.994809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 293/574] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:26.994809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.041421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a009df-79c1-432d-903d-70d117d0d5be None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.049095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.067650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.079426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.079964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.080281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.080546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.081170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:27.081473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.083401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.094545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 282/575] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:27.095935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.116176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c7546-f6a5-4eae-8171-21ca7a1d9a02 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.158528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.159162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.159441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.159714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.172133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:27.177425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:27.178425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:27.179282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 294/576] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:27.179987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.215690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80cfec-6857-4823-9f35-0d423539e472 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.251920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.252578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.252851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.253125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.253757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:27.254089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.255500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.267821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 283/577] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:27.267821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.294009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619bd8a-d901-44d9-bc28-561a4382e32e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.300130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.312718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.332312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.332838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.333119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.333385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.334059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:27.334365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.335725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.354314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 295/578] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:27.354989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.388409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9723e-2e8b-409f-83e1-08d2575e37b5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.425096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.425723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.426031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.426301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.437105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.438766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:27.455386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:27.456408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:27.457345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:27.457345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:27.457345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:27.457345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 284/579] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:27.457345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.472521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a952d4-a41c-4c69-80d6-917d39184da3 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.510282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.511958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.512336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.512728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.513954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:27.514830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.516291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.525410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 296/580] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:27.530845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.555727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.575977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd576b5-cfb4-45d9-9056-d8c453b69337 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.578064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.611142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.611799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.612146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.612485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.619417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.620949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:27.632779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:27.634068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:27.635282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:27.641256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 285/581] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:27.641256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.650999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c35e8-5452-45fa-a0ba-d2934d7319a7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.689418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.690064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.690353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.690608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.699020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.700436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:27.712868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:27.714181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:44:27.715491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:27.718738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 297/582] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:27.719590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.744794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d658b36-9b2e-48c3-a376-a63209d07a1c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.780593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.781529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.781945 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.782168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.782936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:27.783235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.784595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.793889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.803228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 286/583] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:27.804153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.826789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:27.831030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227cfc1-dd4e-4fac-bc44-e66b29b5293c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.866819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.867362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.867715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.867968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.880057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.881688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:27.896335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:27.897312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:27.898243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:27.898243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:27.898243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:27.898243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 298/584] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:27.898243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:27.918317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d4375-bcd9-4ee6-8600-27284b79bcfe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:27.955435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:27.956724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:27.957457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:27.957947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:27.959418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:27.959732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:27.961168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:27.973660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 287/585] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:27.978867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.013141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a25e4e-e7c0-45a1-a053-39a62ef2d20a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.046247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.048603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.049215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.049496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.049802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.050537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:28.050862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.052273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.070990 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 299/586] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:28.071839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.086310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.096029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4065f41-5c80-4719-93a8-314038148fa7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.142616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.143279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.143629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.143973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.154352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.155860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:28.167161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:28.168130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:28.169106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 288/587] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:28.169750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.192849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59132042-962f-4f13-89e8-85fa80b6a773 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.239554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.240161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.240484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.240936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.241404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:28.241707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.243082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.254739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 300/588] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:28.254739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.286571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d63b78-7776-4207-be5a-9d3af9d1d642 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.308066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.322411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.323068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.323863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.324136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/345c823f8d85402ba3b07f39ed5e2fca/roles/ce5c9f6a8b7c4522ad3a175ef1333656` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.339304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=345c823f8d85402ba3b07f39ed5e2fca, role_id=ce5c9f6a8b7c4522ad3a175ef1333656)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.340806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.341292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.354838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 289/589] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:28 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/345c823f8d85402ba3b07f39ed5e2fca/roles/ce5c9f6a8b7c4522ad3a175ef1333656 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:28.355994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.393091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15b94e-352c-4084-abdb-0bcc449a9e3e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.438506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.439160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.439441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.439723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.457699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.459806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:28.481908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:28.482972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:28.483902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:28.483902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:28.483902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:28.486179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 301/590] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:28.487347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.521350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96101e0d-dc8f-4232-b51d-93861c8258c8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.561720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.562393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.563013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.563285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.564014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:28.564321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.565741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.585480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 290/591] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:28.585870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.614566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30afbb9-13e6-4f1d-960c-566fcbd57feb None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.618217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.640526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.654068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.654624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.654905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.655178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.663375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.665090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:28.678572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:28.679621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:28.680592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 302/592] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:28.681211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.717500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a1cec-f133-4a5c-9747-e09545ff18d3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.754134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.754835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.755122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.755290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.755902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:28.756206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.757651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.771518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 291/593] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:28.776793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.821701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5baa4-996f-4686-8e8e-20819e5e245b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.872502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.873534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.874031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.874620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.878257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.888393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.890312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.895873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f852b-872a-44bc-b1d3-1c34ed51a56e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:28.908588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:28.910050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:28.911401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:28.911401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:28.911401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:28.911401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 303/594] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:28.911401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:28.949262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:28.950585 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:28.951261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:28.951681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:28.952808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:28.953925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:28.955830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:28.957498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:28.977347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 292/595] 10.222.0.88 () {56 vars in 1139 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:28.978438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.036393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e9948-22cd-481c-b939-79e87516770a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.075010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4f67-fe36-4a20-89b2-7cfcbd5cc0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.076070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4f67-fe36-4a20-89b2-7cfcbd5cc0de None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.076625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4f67-fe36-4a20-89b2-7cfcbd5cc0de None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.077452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4f67-fe36-4a20-89b2-7cfcbd5cc0de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.081842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d289-a2d7-4e2f-a346-c5d0fb5f991f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.084794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4f67-fe36-4a20-89b2-7cfcbd5cc0de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.086318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4f67-fe36-4a20-89b2-7cfcbd5cc0de None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-a7cd4f67-fe36-4a20-89b2-7cfcbd5cc0de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:29.098522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:29.099491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:29.100430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 304/596] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:29.100925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea67638-a11a-4d05-beb2-3591f5b89758 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.101966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea67638-a11a-4d05-beb2-3591f5b89758 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.101966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea67638-a11a-4d05-beb2-3591f5b89758 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.106189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 305/597] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:29.106189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea67638-a11a-4d05-beb2-3591f5b89758 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.129342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.130149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.130527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.130900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.131663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:29.132032 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.133473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.145457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 293/598] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:29.146429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.158675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:29.217712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:29.226705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea67638-a11a-4d05-beb2-3591f5b89758 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.260178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f43d86-ad6e-4174-9b09-526f43e9dda7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.264441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf89ca-f8dd-4805-b86b-4b6c3f61996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.264932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf89ca-f8dd-4805-b86b-4b6c3f61996c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.265214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf89ca-f8dd-4805-b86b-4b6c3f61996c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.265487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf89ca-f8dd-4805-b86b-4b6c3f61996c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.275716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf89ca-f8dd-4805-b86b-4b6c3f61996c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.277534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf89ca-f8dd-4805-b86b-4b6c3f61996c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-8bbf89ca-f8dd-4805-b86b-4b6c3f61996c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:29.292599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:29.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:29.294723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:29.294723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:29.294723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:29.297932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 306/599] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:29.299583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0a4a-cbf2-42ca-8fd6-1ac0b97b7117 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.300039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0a4a-cbf2-42ca-8fd6-1ac0b97b7117 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.300460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0a4a-cbf2-42ca-8fd6-1ac0b97b7117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.302598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.303210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.303494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.303754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.304445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:29.304756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.306319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.321706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-cf8a0a4a-cbf2-42ca-8fd6-1ac0b97b7117 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:29.327825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 294/600] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:29.327989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cf8a0a4a-cbf2-42ca-8fd6-1ac0b97b7117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:29.328074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.399920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 307/601] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:29.401494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a0a4a-cbf2-42ca-8fd6-1ac0b97b7117 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.407681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:29.444563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93b68c-f9cf-4389-bc59-bc6d2cde7c65 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.485914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.486551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.486838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.487110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.496380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.497871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.503293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:29.513854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:29.514819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 295/602] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:29.516415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.537243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a0a4a-cbf2-42ca-8fd6-1ac0b97b7117 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.585705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.587292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.588047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.588546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.590278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:29.590800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.592881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.609517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 308/603] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:29.610652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.643250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315633-c1a3-4b6a-be6b-ec9c0514db2b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.673710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:29.690824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.692560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.693354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.693888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/37099ac7c39948f0afd272a2b41949ee/roles/ce5c9f6a8b7c4522ad3a175ef1333656` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.712312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=37099ac7c39948f0afd272a2b41949ee, role_id=ce5c9f6a8b7c4522ad3a175ef1333656)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.713866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.726613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 296/604] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:29 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/37099ac7c39948f0afd272a2b41949ee/roles/ce5c9f6a8b7c4522ad3a175ef1333656 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:29.735365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95253028-3fe9-4436-84bc-4dfd7779c970 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.735597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.787419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.788447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.788950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.789659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.791680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:29.792808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.794514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.798810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:29.824207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 309/605] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:29.824207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.891651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1229f-1eb6-41a7-b4c0-97551f7c9de3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.930000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.930789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.931162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.931534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:29.939134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:29.940693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:29.953435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad111a5-6b45-42a3-838c-b25af98f122e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:29.953910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:29.954916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 297/606] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:29.956020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:29.994110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:29.997461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:29.998378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:29.998918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:29.999352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.013703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:30.020049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:30.021223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:30.022340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:30.025626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 310/607] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:30.027276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:30.069357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482e4e4-d604-4305-8545-00746f8f1c00 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:30.100922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:30.112437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:30.113486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.113988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.114442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/6a24bc4a0c4f480192d87a27a243acb1/roles/ce5c9f6a8b7c4522ad3a175ef1333656` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.130568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=6a24bc4a0c4f480192d87a27a243acb1, role_id=ce5c9f6a8b7c4522ad3a175ef1333656)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:30.132395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:30.153372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 298/608] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:29 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/6a24bc4a0c4f480192d87a27a243acb1/roles/ce5c9f6a8b7c4522ad3a175ef1333656 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:30.154598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:30.170102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f9a38-0504-48f9-aacb-92ae7762b308 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:30.251687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:30.252631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.253115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.253630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.254646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:30.255163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:30.261346 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:30.282841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 311/609] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:30.289421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:30.369467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:30.408660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6d86-fe61-46f3-ad46-1541231eb44e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:30.457589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6451e-edcd-4873-a1e1-8e19a82b1e86 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:30.459039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:30.459653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.459927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.460196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.461139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:30.461447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:30.463918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:30.475067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 299/610] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:30.477521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:30.480811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:30.515686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02acf-f1d0-46bf-95e7-f606a3f2a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:30.516625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02acf-f1d0-46bf-95e7-f606a3f2a055 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.517058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02acf-f1d0-46bf-95e7-f606a3f2a055 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.517482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02acf-f1d0-46bf-95e7-f606a3f2a055 None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/dee035c97cb6447382365faf6b6aca53/roles/ce5c9f6a8b7c4522ad3a175ef1333656` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.547954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02acf-f1d0-46bf-95e7-f606a3f2a055 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=dee035c97cb6447382365faf6b6aca53, role_id=ce5c9f6a8b7c4522ad3a175ef1333656)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:30.550002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02acf-f1d0-46bf-95e7-f606a3f2a055 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:30.579574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 312/611] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:30 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/dee035c97cb6447382365faf6b6aca53/roles/ce5c9f6a8b7c4522ad3a175ef1333656 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:44:30.659764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc90e6-ab6e-4dbb-aa57-21e6842fadd2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:30.687660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91289986-d493-452f-80b1-7a86f5064c51 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.687911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91289986-d493-452f-80b1-7a86f5064c51 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.688177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91289986-d493-452f-80b1-7a86f5064c51 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.689905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 313/612] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:30.695765 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c7fe-a9f8-4360-8d46-097ef25cf8ea None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.696050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c7fe-a9f8-4360-8d46-097ef25cf8ea None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.696332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c7fe-a9f8-4360-8d46-097ef25cf8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.711895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:30.712937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.713414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.714450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.743144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:30.745432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-dbb7c7fe-a9f8-4360-8d46-097ef25cf8ea None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:30.746367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:30.750556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-dbb7c7fe-a9f8-4360-8d46-097ef25cf8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:30.766329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:30.780142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:30.781338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:30.782392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:30.782392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:30.782392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:30.784255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 300/613] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:30.790172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:30.868439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 314/614] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:30.931361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736e92f-9d54-4690-842b-32e8cc16e0aa None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:30.969798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:30.970428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:30.970772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:30.971051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:30.971696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:30.972002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:30.973389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:30.987912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 301/615] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:30.993323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7c7fe-a9f8-4360-8d46-097ef25cf8ea None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:31.132559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:31.140011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7c7fe-a9f8-4360-8d46-097ef25cf8ea None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:31.187861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:31.188503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:31.188784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:31.189051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:31.202793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:31.204469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:31.219470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:31.220481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:31.221434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 315/616] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:31.226476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:31.290484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:31.342563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:31.416455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece3afd-d935-4105-a1db-ea5efef15a87 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:31.435254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c7666-f96c-47a5-a333-1c22260bd106 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:31.468350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:31.468965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:31.469244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:31.469515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:31.470279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:31.470591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:31.472017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:31.492274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:31.492911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:31.493198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:31.493475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:31.507507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 302/617] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Sep 19 11:44:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:31.514606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:31.522700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:31.528590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:31.529558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:31.530487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:31.534062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 316/618] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:31.539125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:31.582180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:31.649597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83b2bf-e177-474d-b0ae-60f142383d40 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:31.685142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:31.686094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:31.686553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:31.687002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:31.687845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:31.688492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:31.690055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:31.699689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 317/619] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:31.705957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:31.794255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:31.818145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acbbc1-69cf-4d68-aac1-711332e82a93 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:31.836867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014cd79-4188-4a00-812f-4cabd146479e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:31.857083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:31.858109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:31.858571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:31.859018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:31.871120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:31.872811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:31.890571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:31.891635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:31.892558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:31.892558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:31.892558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:31.893467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:31.895787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 318/620] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:31.902598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:31.969549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:31.970138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:31.970423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:31.970695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:31.988608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:31.999208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.011489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11abfa5-c8db-460e-a497-e91a5ccee237 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:32.027236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:32.029342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:32.031402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:32.031402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:32.031402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:32.034173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 303/621] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 520 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:32.043300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:32.058181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:32.066587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:32.067477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.067936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.068387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.069211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:32.069720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:32.071288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.085074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 319/622] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:32.090705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:32.198255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded393f-991a-4119-9e55-d7e2724c889f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:32.235984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:32.236915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.237374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.237858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.246342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:32.248378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:32.264880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:32.266190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:32.267172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:32.269913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 320/623] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:44:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:32.275653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:32.303069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b3644-7d3e-4188-ad5d-7b82a93001cc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:32.338222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:32.380855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:32.381474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.381751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.382043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.382690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:32.382993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:32.384374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.414992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 304/624] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:32.417728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:32.438967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd0eb1-6ffe-463e-9819-b97efa8e9525 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:32.487553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:32.518144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:32.518752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.519034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.519307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.520039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:32.520343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:32.521784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.562852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 321/625] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:44:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:32.569711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:32.612028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e649a-b9f2-445b-ab8f-31e56c4b3f50 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:32.648534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238fa99-cb7d-47b2-966a-3a1eca5725f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:32.649261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238fa99-cb7d-47b2-966a-3a1eca5725f1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.649612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238fa99-cb7d-47b2-966a-3a1eca5725f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.649992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238fa99-cb7d-47b2-966a-3a1eca5725f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.656985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238fa99-cb7d-47b2-966a-3a1eca5725f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:32.658547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238fa99-cb7d-47b2-966a-3a1eca5725f1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5238fa99-cb7d-47b2-966a-3a1eca5725f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:32.671705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:32.672904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:32.674102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:32.679718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 305/626] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:32.681119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaf764-c6e9-4354-9543-21cf18e249c2 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.681478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaf764-c6e9-4354-9543-21cf18e249c2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.681839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaf764-c6e9-4354-9543-21cf18e249c2 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.688203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 306/627] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:32.689150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaf764-c6e9-4354-9543-21cf18e249c2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:32.694446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa34be-bb06-4895-9be0-a8f6c74cd29f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:32.708008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:32.736068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798cc18-5629-47be-a60b-8d11a4dbf96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:32.736669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798cc18-5629-47be-a60b-8d11a4dbf96e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.736947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798cc18-5629-47be-a60b-8d11a4dbf96e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.737214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798cc18-5629-47be-a60b-8d11a4dbf96e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.797488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:32.809869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798cc18-5629-47be-a60b-8d11a4dbf96e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:32.810052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaf764-c6e9-4354-9543-21cf18e249c2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:32.811315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798cc18-5629-47be-a60b-8d11a4dbf96e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.847483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292489b-813c-4cb9-83bd-b7dbd8588f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:32.848248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292489b-813c-4cb9-83bd-b7dbd8588f0a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.848529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292489b-813c-4cb9-83bd-b7dbd8588f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.849604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292489b-813c-4cb9-83bd-b7dbd8588f0a None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.850355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292489b-813c-4cb9-83bd-b7dbd8588f0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:32.850666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292489b-813c-4cb9-83bd-b7dbd8588f0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:32.852173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292489b-813c-4cb9-83bd-b7dbd8588f0a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:32.871615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 307/628] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:32.871615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caddd43-24e6-4f74-aed4-9a111aff8ff7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.871615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caddd43-24e6-4f74-aed4-9a111aff8ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.871615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caddd43-24e6-4f74-aed4-9a111aff8ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.890318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3caddd43-24e6-4f74-aed4-9a111aff8ff7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:32.891497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3caddd43-24e6-4f74-aed4-9a111aff8ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:32.920206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 322/629] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:32.921262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a53e2-f1ae-4767-adfe-5f5c3b404414 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:32.921545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a53e2-f1ae-4767-adfe-5f5c3b404414 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:32.921953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a53e2-f1ae-4767-adfe-5f5c3b404414 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:32.923331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 323/630] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:32.927915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a53e2-f1ae-4767-adfe-5f5c3b404414 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:32.939600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:32.972740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 308/631] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:32.973591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caddd43-24e6-4f74-aed4-9a111aff8ff7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.036722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a53e2-f1ae-4767-adfe-5f5c3b404414 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.079882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:33.082390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127075f1-b9fa-42d1-a567-35188997e6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.083352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127075f1-b9fa-42d1-a567-35188997e6cf None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.083929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127075f1-b9fa-42d1-a567-35188997e6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.085158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127075f1-b9fa-42d1-a567-35188997e6cf None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/1924693bdf934b15b1a784abfefdea5f/roles/ce5c9f6a8b7c4522ad3a175ef1333656` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.102215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127075f1-b9fa-42d1-a567-35188997e6cf None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=1924693bdf934b15b1a784abfefdea5f, role_id=ce5c9f6a8b7c4522ad3a175ef1333656)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:33.104050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127075f1-b9fa-42d1-a567-35188997e6cf None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:33.108232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caddd43-24e6-4f74-aed4-9a111aff8ff7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.119494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 324/632] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:32 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/1924693bdf934b15b1a784abfefdea5f/roles/ce5c9f6a8b7c4522ad3a175ef1333656 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:33.121642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2786557-afd2-4b4f-b900-8694cde2a880 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.122118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2786557-afd2-4b4f-b900-8694cde2a880 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.122557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2786557-afd2-4b4f-b900-8694cde2a880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.151188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d2786557-afd2-4b4f-b900-8694cde2a880 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:33.152826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d2786557-afd2-4b4f-b900-8694cde2a880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:33.196134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223da54-d3de-4510-955c-1d68c6c5f69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.197113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223da54-d3de-4510-955c-1d68c6c5f69d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.197611 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223da54-d3de-4510-955c-1d68c6c5f69d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.198688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223da54-d3de-4510-955c-1d68c6c5f69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.211389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223da54-d3de-4510-955c-1d68c6c5f69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:33.213154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223da54-d3de-4510-955c-1d68c6c5f69d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:33.220706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7223da54-d3de-4510-955c-1d68c6c5f69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:33.231915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:33.233001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:33.233956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:33.233956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:33.233956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:33.235952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 309/633] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:33.241743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e946ff-1404-424c-8696-5873b7abc6f9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.241743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e946ff-1404-424c-8696-5873b7abc6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.241743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e946ff-1404-424c-8696-5873b7abc6f9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.242407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 325/634] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:33.242503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 310/635] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:33.245442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2786557-afd2-4b4f-b900-8694cde2a880 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.245963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3926ad5-c4ec-43e1-8d65-458d20994ba4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.246444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3926ad5-c4ec-43e1-8d65-458d20994ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.246926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3926ad5-c4ec-43e1-8d65-458d20994ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.283732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f3926ad5-c4ec-43e1-8d65-458d20994ba4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:33.285423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f3926ad5-c4ec-43e1-8d65-458d20994ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:33.371487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2786557-afd2-4b4f-b900-8694cde2a880 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.383517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 311/636] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:33.384373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3926ad5-c4ec-43e1-8d65-458d20994ba4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.419855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.420821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.421249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.421945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.423635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:33.424112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:33.425600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:33.441375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 326/637] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:33.446928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.465443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:33.538967 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3926ad5-c4ec-43e1-8d65-458d20994ba4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.546136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:33.580686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.581355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.581635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.581936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.592698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:33.597631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:33.598655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 312/638] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:33.600053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.615950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82897c74-8d2c-4491-83a9-5e68ed74e0f9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.658498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.660041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.660531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.661008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.670624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:33.672651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:33.689012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:33.690205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:33.691362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:33.693802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 327/639] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:33.695221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.719199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4122ae-9f13-443b-b574-ec67ccc29e90 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.742392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:33.759258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.759898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.760190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.761058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.773967 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:33.778400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:33.780092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:33.782576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 313/640] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:33.785011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.831056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcfbc1-0356-450d-a554-2941839b7252 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.841568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:33.871779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.872387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.872665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.872938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.873599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:33.873933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:33.875297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:33.884246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 328/641] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:33.885044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:33.898116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5670ad-d199-4c95-9fa1-7a3b33da0311 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:33.933599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:33.934411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:33.934523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:33.934702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:33.945493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:33.950862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:33.951881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:33.952772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 314/642] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:33.953806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.002272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3d579-c7a4-4011-9884-00d28eca897a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.003980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:34.044253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.044905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.045243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.045515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.046238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:34.046550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.047967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.066566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 329/643] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:34.066566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.071676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6b39a-8c0e-42d1-9247-550e20feb418 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.097338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:34.108413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.109008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.109285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.109558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.110248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:34.110555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.111964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.121801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 315/644] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:34.122534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.199051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b52846-a087-470b-a8cd-9a10f3e8ae62 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.235439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.236157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.236519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.236914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.247214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba78c0-d06a-40df-9691-f40d049a6889 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.249829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:34.250777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.252658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:34.270867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:34.271936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:34.273022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:34.273022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:34.273022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:34.273022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 330/645] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:34.273261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.294891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.295503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.295783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.296065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.296730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:34.297040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.298430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.308847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 316/646] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:34.310105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.364073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:34.389682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46b8d3-b6e0-4fa2-b48c-a12e40fa24fd None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.413212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4f90-c8a8-4cc2-8967-5d205d39aecb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.427098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.427827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.428103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.428378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.429251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=345c823f8d85402ba3b07f39ed5e2fca) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:34.429624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.431059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.449877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 331/647] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/role_assignments?user.id=345c823f8d85402ba3b07f39ed5e2fca&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:34.449877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.468411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.469225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.469516 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.469871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.482793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.484299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:34.500718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:34.502027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:34.503187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:34.503187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:34.503187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:34.503187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 317/648] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:34.503187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.546169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:34.585230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b89c0-f09c-400a-abb8-942f960fec21 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.628148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95964541-efcd-49ae-a5ef-7a162b775900 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.634902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.636407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.636845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.637944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.638672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:34.638982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.640416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.659480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 332/649] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:34.660698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.672247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.673054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:34.673479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.673997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.674473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.687763 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.689222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:34.707186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:34.708259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:34.709191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:34.709191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:34.709191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:34.709191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 318/650] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:34.709191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.797968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:34.798164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c035-5afa-4f70-8d6b-85b5f49c5c5d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.842805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75592c0-4dcf-47b2-a64d-978da7830ce4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:34.845159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.846847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.847141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.847418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.848071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:34.848375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.849729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.865495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 333/651] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:34.866469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.884294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:34.885264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:34.886459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:34.886964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:34.895677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:34.897697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:34.911581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:34.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:44:34.913977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:34.916273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 319/652] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:34.917744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:34.957865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.006758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3003f-0bfe-4d08-a154-ce1939224c48 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.034674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b543d1-5a3a-4a9a-bb8f-53fc0c9e3a30 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.050815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.051668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.052148 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.052564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.053355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:35.053946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.055534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.072781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.073919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 334/653] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:35.075445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:35.079748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.080393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.080677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.080948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.090435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.092334 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:35.107828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:35.109164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 320/654] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:35.110358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:35.205532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a6f91-552e-4636-973a-1c080d8e10f4 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.236269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.245226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfc62b-27c5-4904-9997-ea8ff4eeff5e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.248021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.248807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.249258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.249678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.250574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:35.251076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.252628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.271473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 335/655] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Sep 19 11:44:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:35.272580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:35.292252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.292885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.293162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.293429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.303463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.305923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:35.320695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:35.321837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 321/656] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:35.322931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:35.388909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.411270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffe244-d9a2-40a0-b961-8b6975084cfb None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.454367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515324bb-866d-4e1f-89c6-675c8a076ad4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.470154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.473910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.474293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.474544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.475246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:35.475557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.477347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.503364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700e04e-5bd8-410a-9c86-907729698d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.504208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700e04e-5bd8-410a-9c86-907729698d01 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.504401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 336/657] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Sep 19 11:44:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:35.504783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700e04e-5bd8-410a-9c86-907729698d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.505196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700e04e-5bd8-410a-9c86-907729698d01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.506186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:35.564891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.589380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700e04e-5bd8-410a-9c86-907729698d01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.590908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700e04e-5bd8-410a-9c86-907729698d01 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.636720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e4b16-45fe-4960-9e46-05b424431c8b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.645358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.676656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.677271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.677694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.677844 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.678579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:35.678885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.680315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.700035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 337/658] 10.222.0.88 () {56 vars in 1139 bytes} [Mon Sep 19 11:44:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:35.700952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:35.712875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 322/659] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:35.712875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e4d0-fa79-4f7a-ab52-2671e5a67d7b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.712875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e4d0-fa79-4f7a-ab52-2671e5a67d7b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.712875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e4d0-fa79-4f7a-ab52-2671e5a67d7b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.712875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 323/660] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:35.712875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299e4d0-fa79-4f7a-ab52-2671e5a67d7b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:35.816378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.820140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb475f-902e-4e74-a97c-36f9c5c6fa4f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.826050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299e4d0-fa79-4f7a-ab52-2671e5a67d7b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:35.864954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.865562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.865870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.866202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.867925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43acea-9cd2-417e-9b2d-09790ee4f43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:35.868060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43acea-9cd2-417e-9b2d-09790ee4f43e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:35.868251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43acea-9cd2-417e-9b2d-09790ee4f43e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:35.868501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43acea-9cd2-417e-9b2d-09790ee4f43e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:35.887855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:35.946837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.948333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:35.951248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43acea-9cd2-417e-9b2d-09790ee4f43e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:35.952678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43acea-9cd2-417e-9b2d-09790ee4f43e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.058852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 324/661] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:36.058852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.071820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.077185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 338/662] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:36.078790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41fa04-9d50-4a5e-bb48-fd3e37c18b76 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.079082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41fa04-9d50-4a5e-bb48-fd3e37c18b76 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.079364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41fa04-9d50-4a5e-bb48-fd3e37c18b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.102068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3f41fa04-9d50-4a5e-bb48-fd3e37c18b76 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:36.103274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3f41fa04-9d50-4a5e-bb48-fd3e37c18b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:36.117302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.182675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22b9b-d4a3-4726-8d8f-49e49d12c0e4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.185655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 339/663] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:36.185929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41fa04-9d50-4a5e-bb48-fd3e37c18b76 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.250267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.250903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.251188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.251456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.262082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.263598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:36.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:36.282097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:36.283322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:36.283322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:36.283322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:36.283322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 325/664] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:36.283322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.327416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41fa04-9d50-4a5e-bb48-fd3e37c18b76 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.359522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.367395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.368043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.368327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.368603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.379849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.381346 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:36.398702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:36.399740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:36.400654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:36.400654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:36.400654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:36.400654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 340/665] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:36.401899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.415458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd21216-7e7e-4b31-97b9-d64035c5355b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.417342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.453009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.454314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.454826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.455296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.467485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.469258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:36.484787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:36.485841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:36.486772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:36.486772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:36.486772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:36.488825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 326/666] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:36.489848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.516027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f209-c44f-4b40-9cff-a0e725b76776 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.552045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.552818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.553094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.553695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.554376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:36.554685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.556029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.572692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 341/667] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:36.572692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.603609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486e98-1f2e-49bf-82ba-822293ec0f34 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.621745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.641415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.642050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.642330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.642608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.643243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:36.643545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.644894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.657068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 327/668] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:36.657915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.675141 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.692758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff965e1-74da-40e2-bb26-60ff7858ce5a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.730839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.731455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.731733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.732001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.732642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:36.733366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.734440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.753264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 342/669] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:36.753264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.776044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e985c-d4d0-4bf7-962d-0ece46281d36 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.812760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.813428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.813711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.814009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.826262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:36.831551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:36.837678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 328/670] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:36.838810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.873947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a8fe9-8339-4c61-9151-816ba1c5b147 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.875917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.911572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.912456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.912876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.913322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.918329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:36.920447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.922735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:36.935187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:36.936176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:36.937108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 343/671] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:36.937624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:36.952398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63f39b-4cf8-4fb8-91a3-4d9aae0e60dd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:36.987015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:36.987638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:36.987925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:36.988201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:36.994790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:36.996221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:37.006965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:37.007900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 329/672] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:37.008822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.039137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7cd0a-e50a-41da-b5ca-42eedfa75537 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.073697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.074321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.074602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.074874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.081816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.083246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:37.094989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:37.095945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 344/673] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:37.096867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.116231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b71ae-a82c-43ee-b554-5919251daf4c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.125892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.153097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102e55-f1cf-4b6f-911a-c7e91be10e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.153697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102e55-f1cf-4b6f-911a-c7e91be10e27 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.154007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102e55-f1cf-4b6f-911a-c7e91be10e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.154277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102e55-f1cf-4b6f-911a-c7e91be10e27 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.154934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3102e55-f1cf-4b6f-911a-c7e91be10e27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.155234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3102e55-f1cf-4b6f-911a-c7e91be10e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.156581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3102e55-f1cf-4b6f-911a-c7e91be10e27 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.157918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.169901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 330/674] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.169901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177120f-464c-4d4f-8516-a5855df2a74e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.169901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177120f-464c-4d4f-8516-a5855df2a74e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.169901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177120f-464c-4d4f-8516-a5855df2a74e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.169901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 331/675] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:37.169901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177120f-464c-4d4f-8516-a5855df2a74e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.212959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3ca5a-4307-4cb2-94c3-7c60b516d9f9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.246365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.246959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.247236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.247506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.248210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.248512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.249938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.262635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 345/676] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.263410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.270355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177120f-464c-4d4f-8516-a5855df2a74e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.304058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.304666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.304952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.305222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.305948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.306258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.307680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.319425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 332/677] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.320226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.347285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.365386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8a796-1d1f-40de-8bbb-5bd0c187d3e0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.376875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.402803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634f2ea-e435-412b-849b-0be0305b8794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.403408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634f2ea-e435-412b-849b-0be0305b8794 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.403685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634f2ea-e435-412b-849b-0be0305b8794 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.403958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634f2ea-e435-412b-849b-0be0305b8794 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.404652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634f2ea-e435-412b-849b-0be0305b8794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.404956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634f2ea-e435-412b-849b-0be0305b8794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.406413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634f2ea-e435-412b-849b-0be0305b8794 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.417693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 346/678] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.419068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400e84e-43e1-414c-96a7-178266e8623d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.419517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400e84e-43e1-414c-96a7-178266e8623d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.419933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400e84e-43e1-414c-96a7-178266e8623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.420784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac2cf4-e2a9-45a5-8381-83f26e73fa52 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.437994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0400e84e-43e1-414c-96a7-178266e8623d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:37.439180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0400e84e-43e1-414c-96a7-178266e8623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:37.456541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.457171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.457449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.457732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.467801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.469229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:37.483483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:37.489534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:37.490668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:37.490668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:37.490668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:37.490668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 333/679] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:37.490668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.508904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 347/680] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:37.509750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400e84e-43e1-414c-96a7-178266e8623d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.585316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.599152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6158d4-4759-4aea-a70c-775943a423b7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.610304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.621756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400e84e-43e1-414c-96a7-178266e8623d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.637421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.638103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.638476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.639164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.639643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=37099ac7c39948f0afd272a2b41949ee) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.640046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.641496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.648800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 334/681] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/role_assignments?user.id=37099ac7c39948f0afd272a2b41949ee&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.650661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.675064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.676041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.676674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.677147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.678177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=dee035c97cb6447382365faf6b6aca53) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.678684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.680431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.689085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 348/682] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/role_assignments?user.id=dee035c97cb6447382365faf6b6aca53&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.690302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.826688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa295-4e5e-4697-b2b0-dd3645d862a0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.868144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.868776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.869056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.869329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.870180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=6a24bc4a0c4f480192d87a27a243acb1) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.870493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.870747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d51e-d822-4d24-8757-611d98e3522a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:37.871983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.881373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 335/683] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/role_assignments?user.id=6a24bc4a0c4f480192d87a27a243acb1&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.882295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:37.918870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:37.919402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:37.919715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:37.920228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:37.920889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:37.921648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:37.923434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:37.926989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.927882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:37.943240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 349/684] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:37.949573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.022470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2532c-85ba-40dc-962a-a14c75864d28 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.075078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.075722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.076002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.076272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.089578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:38.090035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc925305-ace7-4f8e-adf6-a4bf00f88a3d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:38.095213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:38.096253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:38.097130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 336/685] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:38.106795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.140987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.144867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.145403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.145904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] PATH_INFO: `/v3/users/swift` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.157753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:38.159690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:38.174907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:38.175972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:38.176936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:38.179865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 350/686] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:38.188046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.236046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:38.243501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f725c-467f-4f0a-8e68-6f5224905324 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.257113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:38.284105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.284866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.285249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.285624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.286447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:38.286920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:38.288395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:38.302848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 337/687] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:38.304535 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.319510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa47d4f-2404-4c13-a78a-eb69edeb7dff None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.364598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.366008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.366535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.367010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.368084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:38.369114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:38.371383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:38.393076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 351/688] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Sep 19 11:44:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:38.398994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.438869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cc61f-baf1-4226-b473-82b11ceddd75 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.479261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.480360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.480643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.480914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.496042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:38.497504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:38.514425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:38.515483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:38.516440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:38.516440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:38.516440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:38.519463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 338/689] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:38.527741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.544394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9201642-70c2-47b2-9f49-602dc01382f1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.555274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:38.571209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:38.608474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.609097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.609373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.609638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.692306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea3e2-2b5c-4d99-a50f-c5e990e69680 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.737070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:38.739389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:38.740228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.741106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.741600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.742105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.742970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:38.743479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:38.745109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:38.758213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 339/690] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:38.767445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.870118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:38.896555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 352/691] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:38.903246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:38.903448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d58e9e-5a0e-4727-8ff0-728c4720c77a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:38.934144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:38.941845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:38.942541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:38.942767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:38.943042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:38.961019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:38.964276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:38.989548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:38.990664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:38.991635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 340/692] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:44:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:39.002695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:39.054501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4d03-462c-41a3-b581-6e384aab27ca None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:39.092181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:39.092818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:39.093095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:39.093461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:39.104733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:39.106202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:39.122401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:39.123608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:39.124565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:39.124565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:39.124565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:39.124565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 353/693] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:39.134633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:39.170837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9907298-caa1-4d19-8c6c-0da15a467295 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:39.202078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:39.209551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:39.222347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:39.222967 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:39.223248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:39.223623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:39.224363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:39.224673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:39.226245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:39.247380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e31f9-d526-41d1-b146-e41bc9137e7a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:39.269650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 341/694] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:44:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:39.276372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:39.287311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:39.287924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:39.288203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:39.288469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:39.289111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:39.289410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:39.290790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:39.308750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 354/695] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:39.314357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:39.451375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd54029-642b-42c3-a753-d3527b7d3de7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:39.517714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:39.519662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4da6f-83b7-425e-932c-0e38b3490ff9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:39.520308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:39.520587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:39.520856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:39.565490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:39.577031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:39.577656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:39.577955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:39.578222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:39.591610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:39.593256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:39.606585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:39.608634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:39.611070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 355/696] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:39.617787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:39.648931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:39.678829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:39.680300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:39.758596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e84a0-2808-4269-8e00-af4a806fe248 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:39.811945 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 342/697] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:44:39.817674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:39.830692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:39.831301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:39.831577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:39.831881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:39.832569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:39.832869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:39.836146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:39.854748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 356/698] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:39.862699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:39.950268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:39.989845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e8798-069c-4e02-b7fd-27498a5a2aff None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:40.023872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:40.024482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.024764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.025042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.026308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=1924693bdf934b15b1a784abfefdea5f) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:40.026685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:40.028136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:40.033796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:40.042198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 357/699] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:39 2022] GET /identity/v3/role_assignments?user.id=1924693bdf934b15b1a784abfefdea5f&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:40.069394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c621266-6a8e-4595-b18b-c81f4e3751c4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:40.135740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:40.136405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.136713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.137027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.154724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:40.156224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:40.182968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:40.184590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:40.186101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:40.186101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:40.186101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:40.186101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 343/700] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:40.190266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:40.306095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546314e4-bbe4-4f92-96d7-30a2316b53b8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:40.320458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:40.349219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c4d2-5fdd-48a5-bcd6-22396ccd628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:40.349873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c4d2-5fdd-48a5-bcd6-22396ccd628e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.350128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c4d2-5fdd-48a5-bcd6-22396ccd628e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.350410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c4d2-5fdd-48a5-bcd6-22396ccd628e None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.351062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c4d2-5fdd-48a5-bcd6-22396ccd628e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:40.351378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c4d2-5fdd-48a5-bcd6-22396ccd628e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:40.352758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c4d2-5fdd-48a5-bcd6-22396ccd628e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:40.368434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 358/701] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:40.374727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:40.446255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240faaa-5eb1-421b-b30b-bf16f2af0546 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.446600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240faaa-5eb1-421b-b30b-bf16f2af0546 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.446806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240faaa-5eb1-421b-b30b-bf16f2af0546 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.448919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 359/702] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:40.456001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139368b4-a2ab-4152-be7e-97a4a3b1e22b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.456383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139368b4-a2ab-4152-be7e-97a4a3b1e22b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.456585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139368b4-a2ab-4152-be7e-97a4a3b1e22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.484227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-139368b4-a2ab-4152-be7e-97a4a3b1e22b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:40.485926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-139368b4-a2ab-4152-be7e-97a4a3b1e22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:40.516998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7490ba-41f9-4ead-819b-64d7f7ffc6af None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:40.560361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f2f91-6069-46b2-be7b-802ebd92e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:40.560996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2f91-6069-46b2-be7b-802ebd92e72a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.561281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2f91-6069-46b2-be7b-802ebd92e72a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.561556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2f91-6069-46b2-be7b-802ebd92e72a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.569606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f2f91-6069-46b2-be7b-802ebd92e72a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:40.571296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f2f91-6069-46b2-be7b-802ebd92e72a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:40.574522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5a1f2f91-6069-46b2-be7b-802ebd92e72a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:40.587616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:40.589244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:40.590252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:40.590726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 344/703] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:40.591776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf371404-2986-4ebe-88cf-07b6dab8afa0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.592075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf371404-2986-4ebe-88cf-07b6dab8afa0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.592448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf371404-2986-4ebe-88cf-07b6dab8afa0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.597818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 345/704] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:40.598001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 360/705] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:40.606397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ced5-fcfd-4890-8139-29edcdb2755c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.606687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ced5-fcfd-4890-8139-29edcdb2755c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.606957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ced5-fcfd-4890-8139-29edcdb2755c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.612495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139368b4-a2ab-4152-be7e-97a4a3b1e22b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:40.635215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0f40ced5-fcfd-4890-8139-29edcdb2755c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:40.636582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0f40ced5-fcfd-4890-8139-29edcdb2755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:40.779894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 346/706] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:40.799213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:40.803453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139368b4-a2ab-4152-be7e-97a4a3b1e22b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:40.856007 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:40.856682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:40.856972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:40.857419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:40.858090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:40.858401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:40.859879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:40.880672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 361/707] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:40.887403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40ced5-fcfd-4890-8139-29edcdb2755c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:40.906519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.009748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40ced5-fcfd-4890-8139-29edcdb2755c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.045491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:41.048828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652bafb-9833-43ba-945d-2bfc5520dcf7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.055549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fb618-d4ab-490c-b62e-51e054a1cad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.056403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fb618-d4ab-490c-b62e-51e054a1cad6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.056897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fb618-d4ab-490c-b62e-51e054a1cad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.057451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fb618-d4ab-490c-b62e-51e054a1cad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.058373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fb618-d4ab-490c-b62e-51e054a1cad6 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=345c823f8d85402ba3b07f39ed5e2fca) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:41.058854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fb618-d4ab-490c-b62e-51e054a1cad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.060681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fb618-d4ab-490c-b62e-51e054a1cad6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.073976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 347/708] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:40 2022] GET /identity/v3/role_assignments?user.id=345c823f8d85402ba3b07f39ed5e2fca&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:41.075794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a1f8e-e0a0-46bc-84f5-173dde190bd3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.076264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a1f8e-e0a0-46bc-84f5-173dde190bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.076794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a1f8e-e0a0-46bc-84f5-173dde190bd3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.079165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 348/709] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:41.085669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa8a9-7fa1-465d-890e-3fadfaead9c6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.086264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa8a9-7fa1-465d-890e-3fadfaead9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.086698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa8a9-7fa1-465d-890e-3fadfaead9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.102578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.103240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.103518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.103784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] PATH_INFO: `/v3/services/network` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.104960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.106417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.109494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cbfaa8a9-7fa1-465d-890e-3fadfaead9c6 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:41.111155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cbfaa8a9-7fa1-465d-890e-3fadfaead9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:41.128692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:41.130073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:41.131445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 362/710] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:40 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:41.132348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.168057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:41.213932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 349/711] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:41.215296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa8a9-7fa1-465d-890e-3fadfaead9c6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.284216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd243476-0d77-4662-971c-22bf2d70bb61 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.350703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.352090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.352608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.353099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.353960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.356553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.368330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa8a9-7fa1-465d-890e-3fadfaead9c6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:41.369011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:41.370740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:41.371631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:41.375588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 363/712] 10.222.0.88 () {56 vars in 1129 bytes} [Mon Sep 19 11:44:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:41.383032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.402179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:41.427878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.428896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.429384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.430150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.431686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] RBAC: Adding query filter params (name=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:41.432831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.434983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.450510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 350/713] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:44:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:41.456494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.556317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:41.574752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e8a82-730e-4d6e-82d6-75a9d3af3cf3 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.613417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a59caf-bffa-4b1b-9fe2-c55025f3a1cb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.618940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.619935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.620421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.620893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.622196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:41.622714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.624729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.635977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 364/714] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Sep 19 11:44:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:41.635977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.657635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.658680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.659191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.659667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.660508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:41.661015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.662688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.669391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 351/715] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:44:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:41.670660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.732741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:41.764306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba041-4082-4d95-a39d-7d1142996ac4 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.820748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea6aab-22c8-47ba-b077-1612077fb5b0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:41.846794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.847878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.850411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.851062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.879214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.882547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.907281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:41.907988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:41.908094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:41.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:41.909938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:41.909938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:41.909938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:41.910129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:41.910129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:41.910129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:41.910310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 365/716] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:41.910410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:41.911929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:41.917745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:41.922434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 352/717] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Sep 19 11:44:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:41.924118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:42.047496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797a1fe-b2d8-484b-b1a7-1bbe2535ebdb None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:42.091623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a3005-7bee-4e7d-ba9e-fd1de6393ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:42.092258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a3005-7bee-4e7d-ba9e-fd1de6393ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:42.092518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a3005-7bee-4e7d-ba9e-fd1de6393ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:42.092790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a3005-7bee-4e7d-ba9e-fd1de6393ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:42.093451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a3005-7bee-4e7d-ba9e-fd1de6393ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:42.093756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a3005-7bee-4e7d-ba9e-fd1de6393ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:42.095191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a3005-7bee-4e7d-ba9e-fd1de6393ee7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:42.116706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 366/718] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:42.125337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:42.250149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf07f1-cf0e-4c47-adcd-d407ca60bdca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:42.270606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:42.295996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4811708-14bf-4e56-86c4-5b20ba12837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:42.297040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811708-14bf-4e56-86c4-5b20ba12837f None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:42.297543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811708-14bf-4e56-86c4-5b20ba12837f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:42.298057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811708-14bf-4e56-86c4-5b20ba12837f None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:42.298914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4811708-14bf-4e56-86c4-5b20ba12837f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:42.300609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4811708-14bf-4e56-86c4-5b20ba12837f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:42.329900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 353/719] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:44:42 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:43.136039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb50c3-b0fe-4f9e-bc2b-74074a9ffb78 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:43.136583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb50c3-b0fe-4f9e-bc2b-74074a9ffb78 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:43.136583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb50c3-b0fe-4f9e-bc2b-74074a9ffb78 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:43.137931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 367/720] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:43.152252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37c86b-9abe-4a2f-aa48-128fc443bdd5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:43.152693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37c86b-9abe-4a2f-aa48-128fc443bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:43.152806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37c86b-9abe-4a2f-aa48-128fc443bdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:43.190812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5b37c86b-9abe-4a2f-aa48-128fc443bdd5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:43.192374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b37c86b-9abe-4a2f-aa48-128fc443bdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:43.287948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 354/721] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:43.725906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584560d-4574-4a1e-96ee-ffd7564c434c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:43.726279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584560d-4574-4a1e-96ee-ffd7564c434c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:43.726480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584560d-4574-4a1e-96ee-ffd7564c434c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:43.727972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 368/722] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:43.733345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a4b93-734a-4459-ad94-b497974a1bb4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:43.733628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a4b93-734a-4459-ad94-b497974a1bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:43.733938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a4b93-734a-4459-ad94-b497974a1bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:43.737293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584560d-4574-4a1e-96ee-ffd7564c434c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:43.773787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b17a4b93-734a-4459-ad94-b497974a1bb4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:43.774982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b17a4b93-734a-4459-ad94-b497974a1bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:43.858892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:43.919633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 355/723] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:43.930600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584560d-4574-4a1e-96ee-ffd7564c434c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:43.977359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:43.977949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:43.978230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:43.978504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:43.989840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:43.995328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:43.996296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:43.997158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 369/724] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:44.004181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a4b93-734a-4459-ad94-b497974a1bb4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:44.049411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:44.137734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a4b93-734a-4459-ad94-b497974a1bb4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:44.173091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4306f2-9a72-47ec-9604-3a24debadb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:44.173716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4306f2-9a72-47ec-9604-3a24debadb5b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.173987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4306f2-9a72-47ec-9604-3a24debadb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.174258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4306f2-9a72-47ec-9604-3a24debadb5b None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.174918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4306f2-9a72-47ec-9604-3a24debadb5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:44.175216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4306f2-9a72-47ec-9604-3a24debadb5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:44.176559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4306f2-9a72-47ec-9604-3a24debadb5b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:44.185935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 356/725] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:44.191424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:44.341794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac11a3-f649-4b4e-9710-62ed59f20c30 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:44.373379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:44.393145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:44.393912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.394204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.394481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.396552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62bac-9924-4daf-b664-80953403da26 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.396552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62bac-9924-4daf-b664-80953403da26 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.396552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62bac-9924-4daf-b664-80953403da26 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.397805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 357/726] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:44.407001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:44.408664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:44.415356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655bb56-db7a-4dee-8a58-402444f6ea16 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.415635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655bb56-db7a-4dee-8a58-402444f6ea16 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.415904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655bb56-db7a-4dee-8a58-402444f6ea16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:44.430160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:44.431367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:44.432480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:44.432480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:44.432480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:44.436782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 370/727] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:44.443470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:44.457552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8655bb56-db7a-4dee-8a58-402444f6ea16 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:44.459082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8655bb56-db7a-4dee-8a58-402444f6ea16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:44.565081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 358/728] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:44:44.566248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655bb56-db7a-4dee-8a58-402444f6ea16 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:44.571477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71de33-0a81-4ab5-84b5-69e202112974 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:44.608538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:44.612210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb05144-349f-4cb1-81ee-274d0184da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:44.613086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb05144-349f-4cb1-81ee-274d0184da2d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.613562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb05144-349f-4cb1-81ee-274d0184da2d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.614083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb05144-349f-4cb1-81ee-274d0184da2d None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.614924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb05144-349f-4cb1-81ee-274d0184da2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:44.615442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb05144-349f-4cb1-81ee-274d0184da2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:44.617098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb05144-349f-4cb1-81ee-274d0184da2d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:44.631728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 371/729] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:44.633409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e71b-200f-4a18-a05e-44448f037fdb None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.633914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e71b-200f-4a18-a05e-44448f037fdb None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.634402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e71b-200f-4a18-a05e-44448f037fdb None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.636536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 372/730] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:44.637879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84e71b-200f-4a18-a05e-44448f037fdb None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:44.683668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:44.719670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655bb56-db7a-4dee-8a58-402444f6ea16 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:44.754491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84e71b-200f-4a18-a05e-44448f037fdb None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:44.756567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a801aaf-0a93-4e7d-8a3e-c48ce2bc2961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:44.757187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a801aaf-0a93-4e7d-8a3e-c48ce2bc2961 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.757470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a801aaf-0a93-4e7d-8a3e-c48ce2bc2961 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.757746 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a801aaf-0a93-4e7d-8a3e-c48ce2bc2961 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.769537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a801aaf-0a93-4e7d-8a3e-c48ce2bc2961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4a801aaf-0a93-4e7d-8a3e-c48ce2bc2961 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:44.774603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:44.775647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:44.777020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 359/731] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:44.778244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b74ae7-32db-48ca-8973-7e3ee017063c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.778244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b74ae7-32db-48ca-8973-7e3ee017063c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.778413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b74ae7-32db-48ca-8973-7e3ee017063c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.797624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-54b74ae7-32db-48ca-8973-7e3ee017063c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:44.798596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:44.798840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-54b74ae7-32db-48ca-8973-7e3ee017063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:44.799738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.800240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.800749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:44.808606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:44.810393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:44.823642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:44.824671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:44.825925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:44.828011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 373/732] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:44.829213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:44.878629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 360/733] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:44:44.880232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b74ae7-32db-48ca-8973-7e3ee017063c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:44.905016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:44.949519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:44.959635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c90b1-0527-4516-a17b-7a8416614cce None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:44.998691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:44.999295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:44.999574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:44.999847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.000775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:45.000886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.002210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.011085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 374/734] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:45.012050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.033831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b74ae7-32db-48ca-8973-7e3ee017063c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.080154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.081704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.082199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.082892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.084064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:45.084948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.086882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.111120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 361/735] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Sep 19 11:44:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:45.112313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.133097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf3ca-9aa3-4c16-9fc0-623e7faa6006 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.173131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.173775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.174073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.174339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.174992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.176344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.205830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 375/736] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:44:45 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:45.205830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.215092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:45.236589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:45.272734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96026194-ac30-4ab0-b635-bbee1fa15c1b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.334252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.335176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.335455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.335719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.340898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8981-3742-42b4-b533-fae7a3560a56 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.347044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.348525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:45.366725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:45.371964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:45.374856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:45.374856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:45.374856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:45.374856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 362/737] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:45.374856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.389086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb98163-7b08-446d-b80c-f621a59094ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.389992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb98163-7b08-446d-b80c-f621a59094ae None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.390276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb98163-7b08-446d-b80c-f621a59094ae None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.390556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb98163-7b08-446d-b80c-f621a59094ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.469630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb98163-7b08-446d-b80c-f621a59094ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.471211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb98163-7b08-446d-b80c-f621a59094ae None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.479462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ab8dc-da47-42f8-954f-73c133ce02ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.492268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:45.502028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:45.516480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de8edc-f767-4916-a6a3-983b447cd1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.517104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8edc-f767-4916-a6a3-983b447cd1a9 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.517382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8edc-f767-4916-a6a3-983b447cd1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.517657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8edc-f767-4916-a6a3-983b447cd1a9 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.518349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de8edc-f767-4916-a6a3-983b447cd1a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.519716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de8edc-f767-4916-a6a3-983b447cd1a9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.536356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 363/738] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:44:45 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:45.537587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea78dbb-621b-43ea-bf46-765119e1698d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.538087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea78dbb-621b-43ea-bf46-765119e1698d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.538214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea78dbb-621b-43ea-bf46-765119e1698d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.541860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 364/739] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:45.541860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea78dbb-621b-43ea-bf46-765119e1698d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.600457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 376/740] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:45 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:44:45.602198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495e12f-706c-4107-8f61-5cb2cb5c643b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.602678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495e12f-706c-4107-8f61-5cb2cb5c643b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.603179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495e12f-706c-4107-8f61-5cb2cb5c643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.627807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-9495e12f-706c-4107-8f61-5cb2cb5c643b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:45.629048 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9495e12f-706c-4107-8f61-5cb2cb5c643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:45.690650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea78dbb-621b-43ea-bf46-765119e1698d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.717919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 377/741] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:45.719240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495e12f-706c-4107-8f61-5cb2cb5c643b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.740136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.740750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.741027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.741298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.741978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:45.743797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.743797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.761381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 365/742] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:45.764346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.790277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:45.813191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:45.852733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495e12f-706c-4107-8f61-5cb2cb5c643b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.892681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.893316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.893591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.893882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.903118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d799250-0c91-43f5-9f2b-c7e45a5b4f0b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:45.910167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.912406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:45.932405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:45.933436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:45.934399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:45.934399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:45.934399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:45.936237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 378/743] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:45.940507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:45.957062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:45.958637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:45.959127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:45.959632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:45.968170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:45.970165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:45.983207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:45.984214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:45.985189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:45.987753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 366/744] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:44:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:45.993437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.065487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.068512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0a94d-4613-4b34-9bb8-92bc4fcd49ae None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.101243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.105105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.105721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.106032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.106376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.106970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:46.107264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.108614 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.111304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947a982-58f0-4877-beb1-3e11f212342a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.120719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 379/745] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:46.131293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.152005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.152919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.153408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.153944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.154877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:46.155399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.157059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.177184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 367/746] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:44:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:46.183024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.241201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6a9cd-6397-4bdd-b87b-73606ee7448d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.276233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.276983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.277269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.277543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.288407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:46.293054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68485dd8-42ef-4897-9f1d-658f38b0da80 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:46.294282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:46.295256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:46.296132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 380/747] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:46.296982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.333732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.334143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.334953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.335457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.335954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.350120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.351777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:46.368191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:46.369424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:46.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:46.373937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 368/748] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:46.375140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.406890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.421007 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36484ae9-dd30-431d-a697-844a66e218bc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.467831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.468575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.468858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.469282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.481608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.483072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:46.504870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:46.506199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:46.507547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:46.507547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:46.507547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:46.507547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 381/749] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:46.507547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.510450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d266077-b725-41ee-b041-99a6d339f36b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.554841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.556333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.557342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.558194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.559088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:46.559901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.561822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.578434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 369/750] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:46.578434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.595444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.631033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e2e32-e921-46e3-a227-cf9f6b8ca710 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.672788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.673403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.673686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.673989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.674686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:46.674994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.676689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.679394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.699969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 382/751] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:46.701226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.707032 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0f2e-8c29-40d7-b24b-1ed7020ce10e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.755091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.755951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.756483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.756902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.757825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:46.758299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.759809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.774110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 370/752] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:46.775250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.825113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656c06a-9ed7-4b4f-af6b-b42f245976c9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.863879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.870716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.871502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.871880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.872248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.886650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.887494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bbf34-cb47-480a-a13f-1398770f5cb5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:46.888314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:46.907175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:46.908328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:46.909248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:46.909248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:46.909248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:46.913666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 383/753] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:46.914541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:46.933867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:46.934963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:46.935582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:46.936107 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:46.937059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=1924693bdf934b15b1a784abfefdea5f) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:46.937585 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:46.939413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:46.943043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:46.954015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 371/754] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/role_assignments?user.id=1924693bdf934b15b1a784abfefdea5f&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:46.955345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:47.038695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47615dfb-5ef7-4398-b781-6a5e2231274c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:47.080784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:47.081459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.081751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.082300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.090326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:47.092656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:47.107223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:47.108241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:47.109189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 384/755] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:47.117288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:47.194401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:47.202699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8334b-df6a-4f54-9092-9baa2b613db9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:47.233819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5825529-2476-4d73-a9a6-40514b2dce7c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:47.264724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:47.265522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.265949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.266604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.267355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:47.267760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:47.269404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:47.289409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 372/756] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:47.291319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:47.314158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:47.315373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.315804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.316246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.317202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:47.317719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:47.319415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:47.333805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 385/757] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:47.340065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:47.362236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:47.415499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1308c44-1a41-4c8a-878e-5b7b680bb3cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:47.455495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12716f-6057-4993-89a6-e1c350dff898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:47.456407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12716f-6057-4993-89a6-e1c350dff898 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.456785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12716f-6057-4993-89a6-e1c350dff898 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.457153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12716f-6057-4993-89a6-e1c350dff898 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.473938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12716f-6057-4993-89a6-e1c350dff898 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:47.478876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12716f-6057-4993-89a6-e1c350dff898 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bc12716f-6057-4993-89a6-e1c350dff898 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:47.500962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:47.502209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:44:47.503446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:47.505435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 373/758] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:44:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:47.507054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f90c57-612d-4082-9ded-7ffb84dc5da6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.507514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f90c57-612d-4082-9ded-7ffb84dc5da6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.508047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f90c57-612d-4082-9ded-7ffb84dc5da6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.509803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 374/759] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:47.511976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f90c57-612d-4082-9ded-7ffb84dc5da6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:47.521376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5731117-b31e-4a88-be37-80592e3cda86 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:47.524696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:47.560903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96a536-1bef-447d-bc85-d0a9f026db76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:47.561591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a536-1bef-447d-bc85-d0a9f026db76 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.561896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a536-1bef-447d-bc85-d0a9f026db76 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.562167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a536-1bef-447d-bc85-d0a9f026db76 None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/345c823f8d85402ba3b07f39ed5e2fca/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.564203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96a536-1bef-447d-bc85-d0a9f026db76 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=345c823f8d85402ba3b07f39ed5e2fca, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:47.566125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96a536-1bef-447d-bc85-d0a9f026db76 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:47.583118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 386/760] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:47 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/345c823f8d85402ba3b07f39ed5e2fca/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:47.584617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20bbe7-ea86-4d9b-8142-c59cb1008ed8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.584938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20bbe7-ea86-4d9b-8142-c59cb1008ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.585218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20bbe7-ea86-4d9b-8142-c59cb1008ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.606430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4b20bbe7-ea86-4d9b-8142-c59cb1008ed8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:47.607654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4b20bbe7-ea86-4d9b-8142-c59cb1008ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:47.687046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f90c57-612d-4082-9ded-7ffb84dc5da6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:47.742877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:47.775001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:47.775746 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.776028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.776328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:47.777130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:47.777443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:47.778991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:47.814690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 375/761] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Sep 19 11:44:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:47.827638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:47.878759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 387/762] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:47.945856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217f7f0-39b0-4c41-a132-938e862d8c38 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:47.996866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:47.997824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:47.998248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:47.998654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.064027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:48.088668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:48.090066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:48.214582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 376/763] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:44:48.220412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20bbe7-ea86-4d9b-8142-c59cb1008ed8 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:48.236659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:48.338173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:48.375508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3424f1b-ea32-4701-aecb-d23d835167c1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:48.378064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:48.388184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20bbe7-ea86-4d9b-8142-c59cb1008ed8 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:48.429306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:48.429960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:48.430243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:48.430514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.433169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907e9f3-ecf3-4775-aa0d-18a57e77ac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:48.434006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907e9f3-ecf3-4775-aa0d-18a57e77ac15 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:48.434290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907e9f3-ecf3-4775-aa0d-18a57e77ac15 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:48.435663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907e9f3-ecf3-4775-aa0d-18a57e77ac15 None admin] PATH_INFO: `/v3/services/network` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.437841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907e9f3-ecf3-4775-aa0d-18a57e77ac15 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:48.440041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907e9f3-ecf3-4775-aa0d-18a57e77ac15 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:48.446477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:48.447918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3907e9f3-ecf3-4775-aa0d-18a57e77ac15 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:48.451671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:48.452834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:48.455196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 377/764] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:48 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:48.456891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd90c2-37dd-47c9-871c-1636008366e4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:48.456891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd90c2-37dd-47c9-871c-1636008366e4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:48.456891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd90c2-37dd-47c9-871c-1636008366e4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.456891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 378/765] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:48.463492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37b98-81f6-454c-8073-7e6c878df9fc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:48.464049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37b98-81f6-454c-8073-7e6c878df9fc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:48.464525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37b98-81f6-454c-8073-7e6c878df9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:48.467957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:48.468923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:48.470444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:48.470444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:48.470444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:48.473594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 388/766] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:48.475197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:48.491249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-65d37b98-81f6-454c-8073-7e6c878df9fc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:48.492758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-65d37b98-81f6-454c-8073-7e6c878df9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:48.577431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 379/767] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:48.578433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37b98-81f6-454c-8073-7e6c878df9fc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:48.627421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c3fa6-0bb1-4974-9582-570ff5561744 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:48.698653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:48.699654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:48.700151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:48.700629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.701647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] RBAC: Adding query filter params (name=network) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:48.702211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:48.705094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:48.710372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:48.719253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 389/768] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:44:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:48.725486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:48.747479 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:48.776741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37b98-81f6-454c-8073-7e6c878df9fc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:48.843009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:48.843673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:48.843955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:48.844276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.844923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:48.845227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:48.846617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:48.873175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 380/769] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:48.878497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:48.915890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f100e-2fd4-4292-b6d3-274e97f8d12e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:48.968897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:48.969904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:48.970399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:48.970863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:48.971778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:48.972314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:48.973923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:48.985350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 390/770] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:44:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:48.986674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.009138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553cb51-09e4-4f33-a6a6-edc42f3ae5ee None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.051453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:49.058179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.059100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.059574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.060035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.067589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.069374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.074688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:49.084067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:49.085017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:49.085955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:49.092911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 381/771] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:49.094220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.105700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e21b02-a313-485c-b5a7-a1eaf8f412d2 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.154508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.155803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.156081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.156350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.157077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.158919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:49.172338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:49.173411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:49.179503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 391/772] 10.222.0.88 () {56 vars in 1129 bytes} [Mon Sep 19 11:44:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:49.180356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.213954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3e6f6-21d4-4a98-baad-96a3ec85a228 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.253896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.254907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.255382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.255873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.256782 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] RBAC: Adding query filter params (interface=public, service_id=40e85835df9c4991adb8101521c90638, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:49.257286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.258892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.273241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 382/773] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:44:49 2022] GET /identity/v3/endpoints?service_id=40e85835df9c4991adb8101521c90638&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:49.274474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.306866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368273-6d53-4671-9e91-dfc90e5c2d7b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.344751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.345740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.346273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.346758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.347635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:49.348132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.348713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:49.350566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.359892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:49.372681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 392/774] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:49.373985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.410925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2211-4bcd-4a13-be5d-b9fd2a3a7204 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.451035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.452004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.452502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.452977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.454092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:49.454698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.456297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.466305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 383/775] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Sep 19 11:44:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:49.467475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.499208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb54ac-6c17-48b3-b634-d37a9be3d7f4 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.543291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.543918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.544182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.544451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.545061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.546685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.552288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 393/776] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:44:49 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:49.553105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.608682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed29b7-b9e0-4904-9cb8-476f4bd5c77d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.638283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:49.661832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc0191-b4c3-44a2-bbc1-c080182ab37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.662542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0191-b4c3-44a2-bbc1-c080182ab37b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.662823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0191-b4c3-44a2-bbc1-c080182ab37b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.663411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0191-b4c3-44a2-bbc1-c080182ab37b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.664174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc0191-b4c3-44a2-bbc1-c080182ab37b None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=79ac4874ca5043ac9ef8a47f999cac97) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:49.664485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc0191-b4c3-44a2-bbc1-c080182ab37b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.666771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc0191-b4c3-44a2-bbc1-c080182ab37b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.677479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 384/777] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:49 2022] GET /identity/v3/role_assignments?user.id=79ac4874ca5043ac9ef8a47f999cac97&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:49.678873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442488d3-67c7-4af3-ac11-eaf9c9de554b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.679167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442488d3-67c7-4af3-ac11-eaf9c9de554b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.679445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442488d3-67c7-4af3-ac11-eaf9c9de554b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.686252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:49.686694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 385/778] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:49.689355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb5edf-06df-496a-aa2c-68c499c62e0c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.689659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb5edf-06df-496a-aa2c-68c499c62e0c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.689958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb5edf-06df-496a-aa2c-68c499c62e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.723093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-51cb5edf-06df-496a-aa2c-68c499c62e0c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:49.724788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-51cb5edf-06df-496a-aa2c-68c499c62e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:49.775001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12f515-f999-4b2c-9bd7-a8e7230d3b71 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:49.845022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539478d-5e30-40b6-8711-52b755785bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:49.845534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 386/779] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:44:49.845645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539478d-5e30-40b6-8711-52b755785bfa None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:49.845939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539478d-5e30-40b6-8711-52b755785bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:49.846214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539478d-5e30-40b6-8711-52b755785bfa None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:49.846866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539478d-5e30-40b6-8711-52b755785bfa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:49.847167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539478d-5e30-40b6-8711-52b755785bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:49.848557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539478d-5e30-40b6-8711-52b755785bfa None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:49.859063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 394/780] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Sep 19 11:44:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:49.866993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb5edf-06df-496a-aa2c-68c499c62e0c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:49.986737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb5edf-06df-496a-aa2c-68c499c62e0c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:50.014281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:50.050158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:50.051105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.051537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.051971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.052959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] RBAC: Adding query filter params (interface=public, service_id=80d6f2ffef94459fb67629c0a80482a1, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:50.053273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:50.054813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:50.065784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 387/781] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:44:49 2022] GET /identity/v3/endpoints?service_id=80d6f2ffef94459fb67629c0a80482a1&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:50.066717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697787e-7f46-495a-9d81-a60fa19be268 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.067272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697787e-7f46-495a-9d81-a60fa19be268 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.067750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697787e-7f46-495a-9d81-a60fa19be268 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.069270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 395/782] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:50.071975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:50.075994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c701402-09ef-4a3d-8a3d-814bacca3de5 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.076278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c701402-09ef-4a3d-8a3d-814bacca3de5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.076551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c701402-09ef-4a3d-8a3d-814bacca3de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.108289 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4c701402-09ef-4a3d-8a3d-814bacca3de5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:50.110573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4c701402-09ef-4a3d-8a3d-814bacca3de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:50.201635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 396/783] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:44:50.202528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c701402-09ef-4a3d-8a3d-814bacca3de5 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:50.241393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1896f-e0d4-42e4-b915-016466de2fba None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:50.282291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:50.282910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.283189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.283461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.284081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:50.286050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:50.292457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 388/784] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:44:50 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:50.328920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:50.386479 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c701402-09ef-4a3d-8a3d-814bacca3de5 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:50.455719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:50.456390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.456661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.456931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.480922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:50.485950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:50.486942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:50.487814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 397/785] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:50.499251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:50.540339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:50.562004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:50.633352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9f8f3-ad65-4243-b4ce-3de1145d226c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:50.660877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9d90f-5cc3-4fe6-b699-c4f242256ab1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:50.687019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:50.689581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:50.693424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.693741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.694106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.694737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:50.695046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:50.696452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:50.714027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 389/786] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:50.719592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:50.725396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:50.726070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.726350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.726619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.748009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:50.752965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:50.754472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:50.755840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 398/787] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:50.763743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:50.912320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d769d-c0e7-4372-a1f0-a914db0cbd60 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:50.916767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:50.957442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990289e1-ab8f-4c94-a7eb-8e254df8925c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:50.976116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:50.976752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:50.977029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:50.977300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:50.995375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:50.996821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:51.021886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:51.023288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:51.024495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:51.024495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:51.024495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:51.024495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 390/788] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:51.027011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:51.027811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:51.028180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:51.028578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:51.029259 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:51.029564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:51.030979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.033309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:51.048250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 399/789] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:51.050176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:51.122112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:51.205918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c1781-b709-481f-b3f1-9ba8d5c6f709 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:51.245043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7df012-eff8-4683-97b7-754cc46a0582 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:51.274599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:51.275155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:51.275965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:51.276239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:51.291820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:51.293269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:51.310180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:51.311197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:51.312112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:51.312112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:51.312112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:51.312112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 400/790] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:51.317372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:51.319280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:51.319916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:51.320196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:51.320468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:51.321118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:51.321423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:51.323357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.347146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 391/791] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:51.355120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:51.410126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:51.435268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:51.446386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17c55-e36b-4915-8a99-21ca35f8b5aa None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:51.508327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:51.508934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:51.509213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:51.509485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:51.510678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:51.510983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:51.512336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.537089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 401/792] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:51.545241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:51.595753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c26df-c78d-4f1c-8cfe-e257d218255a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:51.663118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:51.663759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:51.664037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:51.664309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:51.673318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:51.674811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:51.694759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:51.695790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:51.696767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 392/793] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:51.707894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:51.736682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145889-f449-40e9-bf68-0ba3b6d6defe None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:51.813393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:51.814234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:51.814513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:51.814783 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:51.827273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:51.828962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:51.852627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:51.854771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:51.856303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 402/794] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:51.867635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:51.868588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:51.895117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:51.907188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bf102-3721-4519-bd93-1eb56ac77c85 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:51.947425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:51.948033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:51.948314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:51.948590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:51.949327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:51.949636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:51.951500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:51.972551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 393/795] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:44:51.979525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:52.103293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45ac28-3a0a-4ba3-8c2d-9c6bd907dbdd None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:52.137297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040990c9-6cd5-47bb-871e-b7bc8d9ca939 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:52.168290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:52.169330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.169813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.170422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.171898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:52.172417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:52.180370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:52.185269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:52.185956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.186240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.186517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.199382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:52.205758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:52.207342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:52.227275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:52.230020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:52.233136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:52.233136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:52.233136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:52.233136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 394/796] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:52.239044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:52.239450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 403/797] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:44:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:52.252382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:52.368795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798610d-9c9e-4c08-80dc-62aabf597af3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:52.400136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:52.424016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca158cd9-50d8-4b09-a010-0abfb76c5548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:52.424630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca158cd9-50d8-4b09-a010-0abfb76c5548 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.424903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca158cd9-50d8-4b09-a010-0abfb76c5548 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.425167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca158cd9-50d8-4b09-a010-0abfb76c5548 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.425826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca158cd9-50d8-4b09-a010-0abfb76c5548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:52.426131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca158cd9-50d8-4b09-a010-0abfb76c5548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:52.427486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca158cd9-50d8-4b09-a010-0abfb76c5548 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:52.440407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 395/798] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:52.442431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6367e20-086f-455b-bcd7-f44df2109825 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.442715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6367e20-086f-455b-bcd7-f44df2109825 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.442987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6367e20-086f-455b-bcd7-f44df2109825 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.444438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 396/799] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:52.445459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd1f0-7ac5-4dd9-b63c-9b0edba3f3bc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.445732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd1f0-7ac5-4dd9-b63c-9b0edba3f3bc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.446046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd1f0-7ac5-4dd9-b63c-9b0edba3f3bc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.447472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 397/800] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:52.449192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee8a7-a386-4b8a-ae76-d61d2a88ced3 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:52.450938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bf91f-33e2-42e7-ad9a-6850d969d1da None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.451652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bf91f-33e2-42e7-ad9a-6850d969d1da None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.451927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bf91f-33e2-42e7-ad9a-6850d969d1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.470743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b64bf91f-33e2-42e7-ad9a-6850d969d1da None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:52.472246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b64bf91f-33e2-42e7-ad9a-6850d969d1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:52.491738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7e3d-b103-4041-a48e-1cb0d8f45c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:52.492533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7e3d-b103-4041-a48e-1cb0d8f45c69 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.492817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7e3d-b103-4041-a48e-1cb0d8f45c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.493092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7e3d-b103-4041-a48e-1cb0d8f45c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.522737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:52.558324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 398/801] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:52.559596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bf91f-33e2-42e7-ad9a-6850d969d1da None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:52.580518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d7e3d-b103-4041-a48e-1cb0d8f45c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:52.582142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d7e3d-b103-4041-a48e-1cb0d8f45c69 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:52.663109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:52.682755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bf91f-33e2-42e7-ad9a-6850d969d1da None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:52.712203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 404/802] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:44:52.714492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da336e80-cd5b-4fae-a7f2-091021e42b21 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.714773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da336e80-cd5b-4fae-a7f2-091021e42b21 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.715058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da336e80-cd5b-4fae-a7f2-091021e42b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.723004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:52.724061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.724581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.725044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.733304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:52.735310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:52.736761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-da336e80-cd5b-4fae-a7f2-091021e42b21 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:52.738423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-da336e80-cd5b-4fae-a7f2-091021e42b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:52.749274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:52.750355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:52.751286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:52.753976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 399/803] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:52.755111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:52.807103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:52.838420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 405/804] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:52.839562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da336e80-cd5b-4fae-a7f2-091021e42b21 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:52.879932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749ba45-875c-4058-8adb-6659bd216263 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:52.917213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:52.917900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:52.918183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:52.918455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:52.928571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:52.930186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:52.944932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:52.945984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:52.946963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:52.946963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:52.946963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:52.946963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 400/805] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:52.946963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:52.960106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:52.964263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da336e80-cd5b-4fae-a7f2-091021e42b21 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:52.999727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4886bb9-56e3-4efb-a3b9-d96b94d3131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.000405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4886bb9-56e3-4efb-a3b9-d96b94d3131a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.000584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4886bb9-56e3-4efb-a3b9-d96b94d3131a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.000845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4886bb9-56e3-4efb-a3b9-d96b94d3131a None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.001531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4886bb9-56e3-4efb-a3b9-d96b94d3131a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:53.001855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4886bb9-56e3-4efb-a3b9-d96b94d3131a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.003249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4886bb9-56e3-4efb-a3b9-d96b94d3131a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.016369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 406/806] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:53.017881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623b7f3-57fd-45eb-9762-24dbbd8c1576 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.018232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623b7f3-57fd-45eb-9762-24dbbd8c1576 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.018577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623b7f3-57fd-45eb-9762-24dbbd8c1576 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.019983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 407/807] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:53.024606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623b7f3-57fd-45eb-9762-24dbbd8c1576 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.052529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:53.065255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87338441-45bc-4d63-8aca-0b7a8b46e120 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.101168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.101593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.101902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.102177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.114887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:53.120167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:53.121152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 401/808] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:53.122400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.140451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623b7f3-57fd-45eb-9762-24dbbd8c1576 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.175346 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a69a2c-e9dd-44aa-9e3b-8d875dddfac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.175976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a69a2c-e9dd-44aa-9e3b-8d875dddfac8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.176162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a69a2c-e9dd-44aa-9e3b-8d875dddfac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.176436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a69a2c-e9dd-44aa-9e3b-8d875dddfac8 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.177081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a69a2c-e9dd-44aa-9e3b-8d875dddfac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:53.177383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a69a2c-e9dd-44aa-9e3b-8d875dddfac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.178763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a69a2c-e9dd-44aa-9e3b-8d875dddfac8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.190922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 408/809] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:53.192170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54b9b-f773-454a-8847-c9a4f61ace3c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.192462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54b9b-f773-454a-8847-c9a4f61ace3c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.192742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54b9b-f773-454a-8847-c9a4f61ace3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.211178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8ef54b9b-f773-454a-8847-c9a4f61ace3c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:53.212370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8ef54b9b-f773-454a-8847-c9a4f61ace3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:53.224989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba02e10-3d4d-4fa4-96a7-aee1381ef164 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.226063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:53.258768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.259425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.260368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.260647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/1924693bdf934b15b1a784abfefdea5f/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.262589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=1924693bdf934b15b1a784abfefdea5f, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.264022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.276726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 402/810] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:53 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/1924693bdf934b15b1a784abfefdea5f/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:53.277521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.282498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 409/811] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:53.283553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef54b9b-f773-454a-8847-c9a4f61ace3c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.289801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:53.397905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40016c1-b497-4b3f-bba8-c34ccb8c0caf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.404273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef54b9b-f773-454a-8847-c9a4f61ace3c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.436654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.437215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.437491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.437795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] PATH_INFO: `/v3/services/network` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.438523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.440040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.444816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.445737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.446274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.446756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.448294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:53.448837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:53.450210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:53.451253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:44:53.452114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:53.453833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.456755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 403/812] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:53.457688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.468920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 410/813] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:53.470314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.486269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:53.544640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:53.579902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e8aa-a348-4970-a4e5-e1b7bf43f3dc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.602683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f4abe-7477-453a-91e7-2f767a55171f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.615631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.616260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.616542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.616815 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.623698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.625165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:53.637440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:53.638435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:53.639557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 404/814] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:53.640169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.643997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.644790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.645213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.645630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.646451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] RBAC: Adding query filter params (name=network) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:53.646901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.648556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.654658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 411/815] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:53.655920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.746359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c314f-79b0-4c9e-839e-184e59735a23 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.760778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:53.781308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:53.787352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.787989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.788274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.788545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.795127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e3711-6929-470e-b76f-696142904525 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.801921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.803389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:53.818157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:53.819241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:53.820186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:53.820186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:53.820186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:53.820186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 405/816] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:53.820186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.836733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.837372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.837654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.838514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.839095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.840459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:53.850228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:53.851325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:53.854989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 412/817] 10.222.0.88 () {56 vars in 1129 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:53.856011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:53.940792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95625-f03f-494c-b183-285db9e263a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.973354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edafb1-8d2e-4e7f-bfd8-28195a689456 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:53.979867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:53.980475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:53.980750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:53.981464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:53.982207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:53.982513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:53.984765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.000480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 406/818] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:54.001241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.008939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.009454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.009853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.010086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.010735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] RBAC: Adding query filter params (type=network) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:54.011039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.012399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.022199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 413/819] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:44:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:54.022199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.026389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:54.050948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:54.113330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b507a0-872f-46a3-b01a-b34c2eead41e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.141009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eb7af-20b4-4ff4-a145-d846a01bd425 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.152584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.153325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.153604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.153954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.154602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:54.155122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.156511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.173349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 407/820] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:54.173349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.187251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.187852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.188128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.188395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.189018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:54.189318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.190713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.195400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 414/821] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Sep 19 11:44:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:54.200555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.269922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:54.284205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1a195-88a7-4fa5-af9b-266d6309b434 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.288229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:54.315688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496e68c-1986-402f-99e0-7fd0b580da99 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.330655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.331245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.331452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.331713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.332493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=345c823f8d85402ba3b07f39ed5e2fca) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:54.332824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.334309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.349813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 408/822] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:54 2022] GET /identity/v3/role_assignments?user.id=345c823f8d85402ba3b07f39ed5e2fca&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:54.349813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.369707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.370593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.370938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.371349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.372213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.373746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.413795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 415/823] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:44:54 2022] POST /identity/v3/endpoints => generated 337 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:44:54.413795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.550222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f7bd0-9733-4906-8de0-491f1163f124 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.609643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.610289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.610565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.610950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.620708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.622300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.630570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a8c6-eac5-414f-a2b0-a5ed438025b2 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:54.635351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:54.636374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:54.637600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 409/824] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:44:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:54.647009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.664890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:54.686746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.687761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.688268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.688945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.690362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:54.690621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:54.691565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.693851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.700077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 416/825] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Sep 19 11:44:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:54.707724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.794046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb20a-7797-4405-979a-2d5aeeb1cffc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.845828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.847468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.848863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.850532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.852227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:54.853326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.855176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.859859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e44a5-8475-4b8c-870a-84137074e514 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:54.880284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 410/826] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Sep 19 11:44:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:54.886850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:54.904366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:54.905267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:54.905564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:54.905915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:54.906856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:54.908734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:54.937902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 417/827] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:44:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:44:55.003613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:55.029276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d2353-08cc-4821-8bd0-548da5016091 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:55.077541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:55.078835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:55.079426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:55.079928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:55.092870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:55.094742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:55.111176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:55.112232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:55.113202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:55.113202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:55.113202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:55.115590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 411/828] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:55.123983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:55.180626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:55.243163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c2fc-9f2e-4d99-9a07-7db0218c82aa None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:55.284278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:55.284898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:55.285177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:55.285454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:55.286143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:55.286452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:55.287808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:55.305585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 418/829] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:55.307378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:55.418868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac117d-4284-421b-8f2c-78344a3056ef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:55.448379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:55.455524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a7559-6293-4c58-a9ea-d64260f5441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:55.456165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a7559-6293-4c58-a9ea-d64260f5441d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:55.456447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a7559-6293-4c58-a9ea-d64260f5441d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:55.456721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a7559-6293-4c58-a9ea-d64260f5441d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:55.464253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a7559-6293-4c58-a9ea-d64260f5441d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:55.465706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a7559-6293-4c58-a9ea-d64260f5441d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-912a7559-6293-4c58-a9ea-d64260f5441d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:55.594559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:55.595626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:55.596586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 412/830] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:55.602166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:55.662855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cdd98-b356-4628-b6af-b2eb3c97d110 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:55.663168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cdd98-b356-4628-b6af-b2eb3c97d110 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:55.663406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cdd98-b356-4628-b6af-b2eb3c97d110 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:55.664847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 413/831] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:55.669999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4e75f-c190-4ad2-b987-f9893616aad0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:55.670213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4e75f-c190-4ad2-b987-f9893616aad0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:55.670489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4e75f-c190-4ad2-b987-f9893616aad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:55.696605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-20a4e75f-c190-4ad2-b987-f9893616aad0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:55.698311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-20a4e75f-c190-4ad2-b987-f9893616aad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:55.724036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9defd-e0a4-4fde-9b78-c94e5026744a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:55.726482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:55.767569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:55.768186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:55.768469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:55.768746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:55.770047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:55.770361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:55.772087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:55.789834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 419/832] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:55.790040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 414/833] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:44:55.794671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:55.910285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98ebb8-9ed0-49f1-abec-ffe3f1ced27b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:55.948163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:55.948775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:55.949079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:55.949389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/79ac4874ca5043ac9ef8a47f999cac97/roles/ce5c9f6a8b7c4522ad3a175ef1333656` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:55.966011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=79ac4874ca5043ac9ef8a47f999cac97, role_id=ce5c9f6a8b7c4522ad3a175ef1333656)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:55.967611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:55.989988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 420/834] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:44:55 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/79ac4874ca5043ac9ef8a47f999cac97/roles/ce5c9f6a8b7c4522ad3a175ef1333656 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:44:56.066866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4e75f-c190-4ad2-b987-f9893616aad0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:56.179289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4e75f-c190-4ad2-b987-f9893616aad0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:56.201052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:56.216873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:56.217502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:56.217792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:56.218077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:56.228554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:56.233277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:56.234357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:56.235920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 415/835] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:56.240529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:56.257122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:56.360303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76c0ae-4ff0-4942-ae4e-c45805dbd26d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:56.404072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:56.405113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:56.405651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:56.406211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:56.407142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:56.407658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:56.409209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:56.420364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 421/836] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:56.428400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:56.529945 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4e1b-1bd3-42bd-a351-ed3f9463071d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:56.571629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:56.572856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:56.573140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:56.573470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:56.584121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:56.585570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:56.601292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:56.602347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:56.603839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:56.603839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:56.603839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:56.603839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 416/837] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:56.608536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:56.695931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:56.731438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574f2a9-0b6a-4096-bf98-bb3bc481bf40 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:56.768691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaadbc3-0294-40dd-a058-d350761ea3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:56.769319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaadbc3-0294-40dd-a058-d350761ea3be None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:56.769607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaadbc3-0294-40dd-a058-d350761ea3be None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:56.769937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaadbc3-0294-40dd-a058-d350761ea3be None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:56.770594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaadbc3-0294-40dd-a058-d350761ea3be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:56.770940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaadbc3-0294-40dd-a058-d350761ea3be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:56.772407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaadbc3-0294-40dd-a058-d350761ea3be None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:56.785848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 422/838] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:56.791733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:56.802311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843047f9-ac37-4442-84eb-22355e3517ff None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:56.802621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843047f9-ac37-4442-84eb-22355e3517ff None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:56.802867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843047f9-ac37-4442-84eb-22355e3517ff None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:56.804442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 423/839] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:44:56.810016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf47d1-ed95-42d4-9579-9a9ba62ea2db None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:56.810299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf47d1-ed95-42d4-9579-9a9ba62ea2db None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:56.810572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf47d1-ed95-42d4-9579-9a9ba62ea2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:56.827700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:56.830238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2ddf47d1-ed95-42d4-9579-9a9ba62ea2db None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:56.831493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2ddf47d1-ed95-42d4-9579-9a9ba62ea2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:56.906268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 424/840] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:44:56.908920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46884d2-4981-4048-ab89-6cebd19ed738 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:56.947096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:56.947818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:56.948093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:56.948361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:56.956437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:56.957885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:56.970265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:56.971330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:56.973100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 417/841] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:44:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:56.977497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf47d1-ed95-42d4-9579-9a9ba62ea2db None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:57.053812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:57.097938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf47d1-ed95-42d4-9579-9a9ba62ea2db None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:57.141159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:57.141800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.142074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.145110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.145110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:57.145110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:57.145110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:57.163528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 425/842] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Sep 19 11:44:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:57.169590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:57.185737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:57.291567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:57.302009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f7bf-c3d5-4fd2-9540-26dcfd61fed8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:57.303756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0658-e7f9-444c-a08d-a4741e24a44c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:57.332235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:57.347621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55308a-97d8-4c24-8eb1-8696896ee7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:57.348848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55308a-97d8-4c24-8eb1-8696896ee7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.349390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55308a-97d8-4c24-8eb1-8696896ee7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.349686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:57.350155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55308a-97d8-4c24-8eb1-8696896ee7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.350340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.350608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.350877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.351548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:57.353089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:57.363277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:57.364421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:57.365442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 426/843] 10.222.0.88 () {56 vars in 1132 bytes} [Mon Sep 19 11:44:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:57.372416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:57.433881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55308a-97d8-4c24-8eb1-8696896ee7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:57.435434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55308a-97d8-4c24-8eb1-8696896ee7f6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:57.489360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a8b21-9316-4ddb-a92f-391d7cf8ab7c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:57.526702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:57.529543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:57.530257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.530571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.530902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.531553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:57.531856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:57.533330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:57.542015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 418/844] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:57 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:44:57.542015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b24b62-b850-4de0-b804-dbb6ea865164 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.542015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b24b62-b850-4de0-b804-dbb6ea865164 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.542015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b24b62-b850-4de0-b804-dbb6ea865164 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.543910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 427/845] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:44:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:57.544814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 419/846] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:57.545180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:57.545752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b24b62-b850-4de0-b804-dbb6ea865164 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:57.603093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:57.650763 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b24b62-b850-4de0-b804-dbb6ea865164 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:57.658584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f49fd5-8596-44c4-85d2-a57eddf04c57 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:57.688049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d06c75-f806-4273-ae06-ce30f1e67a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:57.688692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d06c75-f806-4273-ae06-ce30f1e67a96 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.688974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d06c75-f806-4273-ae06-ce30f1e67a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.689249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d06c75-f806-4273-ae06-ce30f1e67a96 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.689908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d06c75-f806-4273-ae06-ce30f1e67a96 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:57.690231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d06c75-f806-4273-ae06-ce30f1e67a96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:57.691661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d06c75-f806-4273-ae06-ce30f1e67a96 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:57.696384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 420/847] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:44:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:57.697460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95b556-29e9-4c42-bc41-8fc6878480df None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.697743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95b556-29e9-4c42-bc41-8fc6878480df None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.698041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95b556-29e9-4c42-bc41-8fc6878480df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.713307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:57.714792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.715428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.715900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.718920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9c95b556-29e9-4c42-bc41-8fc6878480df None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:57.720363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9c95b556-29e9-4c42-bc41-8fc6878480df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:57.733092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:57.735364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:57.751878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:57.752951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:57.753949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:57.753949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:57.753949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:57.757995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 428/848] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:57.758998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:57.796517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:57.822256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 421/849] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:57.889519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e95a0a-3565-4019-ba50-1850201e5c26 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:57.891191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:57.929680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:57.930316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:57.930590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:57.930857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:57.935522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:57.935882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:57.937344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:57.952944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 429/850] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:57.963612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95b556-29e9-4c42-bc41-8fc6878480df None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:58.080858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95b556-29e9-4c42-bc41-8fc6878480df None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:58.116835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:58.119667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:58.120305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.120581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.120851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.128206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:58.129666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:44:58.142015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:58.142978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:58.143912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 422/851] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:44:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:58.150435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:58.198742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:58.238747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a580080-f2fb-4f19-abaf-d09d5f00fd1c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:58.255366 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba38c2-e32b-4363-98a2-a98c45fbe7cf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:58.276442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:58.277454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.277980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.278453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.279397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:58.280958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:58.290701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:58.291755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:44:58.292616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:44:58.293400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:58.293400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.293400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.293400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.293691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:58.293691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:58.294567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:58.300919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 430/852] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:44:58.302146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:58.312426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 423/853] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:44:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:58.315645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:58.343769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:58.447772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8d3de-18b2-4451-ab2c-a01a7ede2f71 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:58.470776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa7857-8996-4398-ab37-ce65a95692a9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:58.479907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:58.488190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f0842-43fb-47bb-9303-9fe3e7cc03b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:58.488799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f0842-43fb-47bb-9303-9fe3e7cc03b8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.489078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f0842-43fb-47bb-9303-9fe3e7cc03b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.489369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f0842-43fb-47bb-9303-9fe3e7cc03b8 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.489999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f0842-43fb-47bb-9303-9fe3e7cc03b8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:58.490299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f0842-43fb-47bb-9303-9fe3e7cc03b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:58.491634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f0842-43fb-47bb-9303-9fe3e7cc03b8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:58.501492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 431/854] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:58.501492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106e5c-2a0e-4090-81ed-04b088204000 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.501492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106e5c-2a0e-4090-81ed-04b088204000 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.501492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106e5c-2a0e-4090-81ed-04b088204000 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.501492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 432/855] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:44:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:44:58.502443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106e5c-2a0e-4090-81ed-04b088204000 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:58.516750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb39aa-d439-4847-b774-7b0ca6317087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:58.517363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb39aa-d439-4847-b774-7b0ca6317087 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.517645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb39aa-d439-4847-b774-7b0ca6317087 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.517983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb39aa-d439-4847-b774-7b0ca6317087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.518719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb39aa-d439-4847-b774-7b0ca6317087 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=1924693bdf934b15b1a784abfefdea5f) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:58.519042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb39aa-d439-4847-b774-7b0ca6317087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:58.520467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb39aa-d439-4847-b774-7b0ca6317087 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:58.526612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 424/856] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:44:58 2022] GET /identity/v3/role_assignments?user.id=1924693bdf934b15b1a784abfefdea5f&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:58.527802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa348ae-455b-42ac-9b0b-2f9239b8f7c7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.528085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa348ae-455b-42ac-9b0b-2f9239b8f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.528357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa348ae-455b-42ac-9b0b-2f9239b8f7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.548886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dfa348ae-455b-42ac-9b0b-2f9239b8f7c7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:44:58.550210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dfa348ae-455b-42ac-9b0b-2f9239b8f7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:58.630848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:58.634398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 425/857] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:44:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:44:58.650433 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106e5c-2a0e-4090-81ed-04b088204000 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:58.689569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:58.690650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:58.691170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:58.691649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:58.692857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:58.693424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:58.695140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:58.702638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 433/858] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:44:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:58.921179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa348ae-455b-42ac-9b0b-2f9239b8f7c7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:59.025828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:59.029686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa348ae-455b-42ac-9b0b-2f9239b8f7c7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:59.069898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:59.070537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:59.070818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:59.071090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:59.082822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:44:59.088339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:59.089344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:59.090285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 426/859] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:44:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:59.097398 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:59.204653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760a043-f87b-4347-b3e8-7bb6c0c4f116 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:59.239894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:59.240905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:59.241392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:59.241931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:59.242862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:59.243382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:59.245469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:59.254271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 434/860] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:44:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:59.260320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:59.365726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69764c-1c38-492b-bd10-b4e6408942dc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:59.402582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:59.403211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:59.403486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:59.403750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:59.414170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:59.415644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:44:59.430710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:59.432281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:44:59.433239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:44:59.433239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:44:59.433239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:59.433239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 427/861] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:59.439730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:59.466134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:59.549245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f44ec-34bb-439f-b308-9612a1010baa None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:59.586075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:59.587060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:59.587726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:59.588214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:59.589061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:59.589578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:59.591216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:59.605295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 435/862] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:59.611438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:59.650820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:59.718503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef925-eb2b-4ca8-a516-f1bb52896479 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:59.752766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:59.753403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:59.753680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:59.753972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:59.762240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:59.763658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:44:59.774607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:44:59.776162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:44:59.777911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 428/863] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:44:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:44:59.783124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:59.874138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:59.902452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61807d0-5816-410b-b269-20075ef52006 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:59.944157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edb77f-7ee0-4fae-8b45-c3d1c2955164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:59.945076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edb77f-7ee0-4fae-8b45-c3d1c2955164 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:59.945583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edb77f-7ee0-4fae-8b45-c3d1c2955164 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:59.946114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edb77f-7ee0-4fae-8b45-c3d1c2955164 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:59.947381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edb77f-7ee0-4fae-8b45-c3d1c2955164 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:44:59.947930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edb77f-7ee0-4fae-8b45-c3d1c2955164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:59.949538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edb77f-7ee0-4fae-8b45-c3d1c2955164 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:59.971020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 436/864] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Sep 19 11:44:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:44:59.976885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:00.062458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:00.098240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fd83a-8dc9-4a44-9c4d-350bc5f59982 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:00.134757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9c8e-4459-41f1-8d83-213a09c06810 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.135717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9c8e-4459-41f1-8d83-213a09c06810 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.136207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9c8e-4459-41f1-8d83-213a09c06810 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.139022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:00.139322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 437/865] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:00.141809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.141809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.141809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.144101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8c8b-c891-4f10-90ce-de49e5a1a8b6 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.144622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8c8b-c891-4f10-90ce-de49e5a1a8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.145119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8c8b-c891-4f10-90ce-de49e5a1a8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.169379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5f6b8c8b-c891-4f10-90ce-de49e5a1a8b6 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:00.170948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5f6b8c8b-c891-4f10-90ce-de49e5a1a8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:00.223440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:00.225264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:00.257879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 438/866] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:00.310372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:00.351009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 429/867] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:44:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:00.356974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b8c8b-c891-4f10-90ce-de49e5a1a8b6 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:00.430874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:00.476044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b8c8b-c891-4f10-90ce-de49e5a1a8b6 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:00.515776 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:00.516678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.517188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.517610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.529875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:00.531697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:00.537164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:00.550212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:00.551526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:00.552686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:00.552686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:00.552686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:00.554678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 439/868] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:00.563391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:00.677151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ccad0b-8b70-4c6f-84a6-870c257b1e17 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:00.696035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96cc51-fa61-4a45-af13-60fa17ef15e5 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:00.716548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:00.729072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:00.730075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.730362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.730638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.731316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:00.733500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:00.743190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9f7e7-d251-4e2d-b0f4-2328bbaf90c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:00.744501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f7e7-d251-4e2d-b0f4-2328bbaf90c9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.745348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f7e7-d251-4e2d-b0f4-2328bbaf90c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.745822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f7e7-d251-4e2d-b0f4-2328bbaf90c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.746983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9f7e7-d251-4e2d-b0f4-2328bbaf90c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:00.747517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9f7e7-d251-4e2d-b0f4-2328bbaf90c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:00.749063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9f7e7-d251-4e2d-b0f4-2328bbaf90c9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:00.750760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 430/869] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:45:00 2022] POST /identity/v3/services => generated 269 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:45:00.765012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 440/870] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:00.773822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:00.815098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:00.914899 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709803d1-2264-430c-b5d5-cc3e3ae95b1d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:00.950964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51289fc-a7be-4236-8c41-e9ad478b6592 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.951148 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51289fc-a7be-4236-8c41-e9ad478b6592 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.951432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51289fc-a7be-4236-8c41-e9ad478b6592 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.952760 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 441/871] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:00.955599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:00.956480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.956759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.957025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.959094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e49dab-49f1-4e2a-b15f-c178ceaf7dc4 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:00.959379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e49dab-49f1-4e2a-b15f-c178ceaf7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:00.959655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e49dab-49f1-4e2a-b15f-c178ceaf7dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:00.964617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:00.966143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:00.979600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-53e49dab-49f1-4e2a-b15f-c178ceaf7dc4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:00.980288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:00.981196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:00.982165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 431/872] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:00.983360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-53e49dab-49f1-4e2a-b15f-c178ceaf7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:00.989788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:01.062662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 442/873] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:00 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:01.063878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2985b8-f2db-4515-bff3-3b101c8d7739 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.064166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2985b8-f2db-4515-bff3-3b101c8d7739 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.064440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2985b8-f2db-4515-bff3-3b101c8d7739 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.065956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 443/874] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:01.071661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973299-f820-43bc-a3f2-d8d959282152 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.072481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973299-f820-43bc-a3f2-d8d959282152 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.072759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973299-f820-43bc-a3f2-d8d959282152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.097158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:01.098802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-cb973299-f820-43bc-a3f2-d8d959282152 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:01.101116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cb973299-f820-43bc-a3f2-d8d959282152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:01.112022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e07ff-66fa-4dfa-b395-e111f8e0b9b2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:01.158208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:01.159306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.159730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.160311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.161437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:01.161925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:01.163523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:01.180143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 432/875] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:01.185814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 444/876] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:01.186411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:01.293808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2133ea-910a-4bfb-914d-d403c094bb4e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:01.332034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:01.332684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.332927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.333179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.333949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=79ac4874ca5043ac9ef8a47f999cac97) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:01.334264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:01.335781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:01.339623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973299-f820-43bc-a3f2-d8d959282152 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:01.343391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 433/877] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:01 2022] GET /identity/v3/role_assignments?user.id=79ac4874ca5043ac9ef8a47f999cac97&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:01.379299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:01.458752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973299-f820-43bc-a3f2-d8d959282152 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:01.459765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:01.480664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:01.499013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:01.499688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.499965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.500234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.500908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:01.502339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:01.519342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 445/878] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:45:01 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:45:01.599177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3bc7-e06c-4e0e-851c-37a09705f03b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:01.646424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:01.647069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.647349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.647651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.648253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:01.649641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:01.659965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:01.661085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:01.662275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 434/879] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:01.668512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:01.773245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:01.780356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641af2-5163-455e-ac8a-380525c3b270 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:01.819116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f6f12-7217-44e3-8655-9859eb26d912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:01.819721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6f12-7217-44e3-8655-9859eb26d912 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.819994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6f12-7217-44e3-8655-9859eb26d912 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.820261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6f12-7217-44e3-8655-9859eb26d912 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.820886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f6f12-7217-44e3-8655-9859eb26d912 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:01.821189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f6f12-7217-44e3-8655-9859eb26d912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:01.823081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f6f12-7217-44e3-8655-9859eb26d912 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:01.828280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 446/880] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:01.833999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:01.909380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:01.941785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ada7f-7439-49d9-9f4f-2dd32638ea68 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:01.978644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c389522-25d7-405d-935f-fdfe88430149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:01.979503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389522-25d7-405d-935f-fdfe88430149 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:01.979921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389522-25d7-405d-935f-fdfe88430149 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:01.980363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389522-25d7-405d-935f-fdfe88430149 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:01.981134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c389522-25d7-405d-935f-fdfe88430149 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:01.981577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c389522-25d7-405d-935f-fdfe88430149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:01.983097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c389522-25d7-405d-935f-fdfe88430149 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:01.988614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 435/881] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:03.102639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b39c-4592-4a56-b6d4-678e6cc9a371 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.103231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b39c-4592-4a56-b6d4-678e6cc9a371 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.103231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b39c-4592-4a56-b6d4-678e6cc9a371 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.104781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 447/882] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:03.110345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4c02-24de-4e0a-80cf-6bc2d5f0d03a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.110797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4c02-24de-4e0a-80cf-6bc2d5f0d03a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.110884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4c02-24de-4e0a-80cf-6bc2d5f0d03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.129585 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-275d4c02-24de-4e0a-80cf-6bc2d5f0d03a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:03.130798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-275d4c02-24de-4e0a-80cf-6bc2d5f0d03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:03.205323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 436/883] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:03.473856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b39c-4592-4a56-b6d4-678e6cc9a371 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:03.580238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b39c-4592-4a56-b6d4-678e6cc9a371 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:03.616446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a8229-14b5-4900-8dee-61cd6e95c266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:03.617504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a8229-14b5-4900-8dee-61cd6e95c266 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.618055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a8229-14b5-4900-8dee-61cd6e95c266 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.618560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a8229-14b5-4900-8dee-61cd6e95c266 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.619459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a8229-14b5-4900-8dee-61cd6e95c266 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:03.621084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a8229-14b5-4900-8dee-61cd6e95c266 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-604a8229-14b5-4900-8dee-61cd6e95c266 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:03.631544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:03.632597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:45:03.633544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:03.637359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 448/884] 10.222.0.88 () {56 vars in 1132 bytes} [Mon Sep 19 11:45:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:03.643246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d4c02-24de-4e0a-80cf-6bc2d5f0d03a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:03.712946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:03.747366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbc39-dc3a-4b24-8317-60402522a54b None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.747551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbc39-dc3a-4b24-8317-60402522a54b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.747832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbc39-dc3a-4b24-8317-60402522a54b None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.749431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 449/885] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:03.751960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d4c02-24de-4e0a-80cf-6bc2d5f0d03a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:03.754639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d90f0-dbd4-474f-bad9-fedcc9ff61a2 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.754926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d90f0-dbd4-474f-bad9-fedcc9ff61a2 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.755206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d90f0-dbd4-474f-bad9-fedcc9ff61a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.781145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8b8d90f0-dbd4-474f-bad9-fedcc9ff61a2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:03.782406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8b8d90f0-dbd4-474f-bad9-fedcc9ff61a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:03.800099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:03.800990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.801399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.801876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.802810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:03.803268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:03.804746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:03.805411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:03.813800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 437/886] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:03.820984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:03.869732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 450/887] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:03.871140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f1606-d960-4893-a018-28e184348d01 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.871424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f1606-d960-4893-a018-28e184348d01 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.871697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f1606-d960-4893-a018-28e184348d01 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.872941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 451/888] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:03.878950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091ad85-219b-495b-a40d-fc943783ef80 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.879213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091ad85-219b-495b-a40d-fc943783ef80 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.879484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091ad85-219b-495b-a40d-fc943783ef80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.900481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8091ad85-219b-495b-a40d-fc943783ef80 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:03.901685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8091ad85-219b-495b-a40d-fc943783ef80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:03.933158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf9d49-49ab-4ead-a61f-3b1e87f4b937 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:03.964033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:03.971797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:03.972411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:03.972692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:03.972966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:03.973606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:03.974056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:03.975643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:03.981397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 438/890] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:03.981564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 452/890] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:03.987131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.095452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80cafe-d440-40cb-be2e-29765f8bd2f8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.133733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.135024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.135341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.135619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.136328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] RBAC: Adding query filter params (interface=public, service_id=6c505f9f3fd146df8caf72e4b57a21ea, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:04.136637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:04.138025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:04.143647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091ad85-219b-495b-a40d-fc943783ef80 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.153003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 439/891] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:45:03 2022] GET /identity/v3/endpoints?service_id=6c505f9f3fd146df8caf72e4b57a21ea&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:04.160118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.227845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:04.252225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:04.273238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091ad85-219b-495b-a40d-fc943783ef80 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.284000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1409e-8bd2-4ec5-a015-bd8f504e646d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.316965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eda562-259c-47dc-b780-1268c7615c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.317593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eda562-259c-47dc-b780-1268c7615c97 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.317913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eda562-259c-47dc-b780-1268c7615c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.318188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eda562-259c-47dc-b780-1268c7615c97 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.318301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b8fe7-0609-46fb-8cb6-f247acbd254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.318891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b8fe7-0609-46fb-8cb6-f247acbd254c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.319172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b8fe7-0609-46fb-8cb6-f247acbd254c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.319444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b8fe7-0609-46fb-8cb6-f247acbd254c None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.320053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b8fe7-0609-46fb-8cb6-f247acbd254c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:04.321418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b8fe7-0609-46fb-8cb6-f247acbd254c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:04.327637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 440/892] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:45:04.328882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4f63e-3973-4ea9-ba15-de4512e8f5ae None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.329158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4f63e-3973-4ea9-ba15-de4512e8f5ae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.329337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eda562-259c-47dc-b780-1268c7615c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:04.329452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4f63e-3973-4ea9-ba15-de4512e8f5ae None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.330809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 441/893] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:04.334380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4f63e-3973-4ea9-ba15-de4512e8f5ae None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-52eda562-259c-47dc-b780-1268c7615c97 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:04.334536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:04.335497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:04.336363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 453/894] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:04.337102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2284d-80e1-43f4-9c6d-ef2c7e460efd None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.337453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2284d-80e1-43f4-9c6d-ef2c7e460efd None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.338278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2284d-80e1-43f4-9c6d-ef2c7e460efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.359706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0aa2284d-80e1-43f4-9c6d-ef2c7e460efd None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:04.360828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0aa2284d-80e1-43f4-9c6d-ef2c7e460efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:04.450418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 454/895] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:04.450418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2284d-80e1-43f4-9c6d-ef2c7e460efd None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.472975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4f63e-3973-4ea9-ba15-de4512e8f5ae None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.505746 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:04.523688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.524462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.525052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.525346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.525991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:04.527352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:04.536942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:04.538078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:04.538977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 442/896] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:04.544619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.564851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:04.601658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2284d-80e1-43f4-9c6d-ef2c7e460efd None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.654110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661280a2-f920-4328-9786-897976435634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.654898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661280a2-f920-4328-9786-897976435634 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.655191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661280a2-f920-4328-9786-897976435634 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.655470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661280a2-f920-4328-9786-897976435634 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.656646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661280a2-f920-4328-9786-897976435634 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:04.657319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661280a2-f920-4328-9786-897976435634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:04.659001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661280a2-f920-4328-9786-897976435634 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:04.671451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 455/897] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:04.677129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661280a2-f920-4328-9786-897976435634 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.680146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e72c-17a9-4486-ba1f-0107aae21efc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.721230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.721867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.722145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.722413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.723045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:04.723345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:04.724706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:04.730115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 443/898] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:04.732289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.783966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:04.802825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661280a2-f920-4328-9786-897976435634 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.841196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.841861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.842161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.842416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.843566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf2428-c578-41a8-aba8-e8726f23f01f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:04.845280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:04.854533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:04.855975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:04.871054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:04.872239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:04.873355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:04.873355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:04.873355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:04.873355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 456/899] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:04.873355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:04.888496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:04.889390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:04.889675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:04.890012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:04.890626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:04.891996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:04.909887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 444/900] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:45:04 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:45:04.910055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:05.013604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b6b4d-87e4-4ea8-b913-5fe7204e810a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.047114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e08ca2-ecc8-42dd-8524-0d918828d7f3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.055698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:05.064106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.064991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.065441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.065905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:05.067031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:05.068078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:05.070053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:05.077118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 457/901] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:05.085051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:05.097678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.097933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.098095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.098365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:05.099019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:05.099321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:05.100774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:05.113424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 445/902] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:05.119007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:05.161380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:05.212036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951e706-21cb-4b2a-a629-e5137ddf71ce None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.239841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15790ab3-1961-4982-bf91-5338720587f6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.249985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.250682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.251150 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.251502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:05.252325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] RBAC: Adding query filter params (interface=public, service_id=5b0889f262f74c0e84cfc5285e62c0e4, region_id=RegionOne) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:05.252706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:05.254288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:05.263535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 458/903] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:45:05 2022] GET /identity/v3/endpoints?service_id=5b0889f262f74c0e84cfc5285e62c0e4&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:05.269587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:05.298132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017367b-7710-4850-a165-49603812600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.298765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017367b-7710-4850-a165-49603812600d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.299383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017367b-7710-4850-a165-49603812600d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.299383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017367b-7710-4850-a165-49603812600d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:05.308137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017367b-7710-4850-a165-49603812600d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:05.309573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017367b-7710-4850-a165-49603812600d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2017367b-7710-4850-a165-49603812600d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:45:05.321477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:05.323070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:05.324042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 446/904] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:45:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:05.329247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017367b-7710-4850-a165-49603812600d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:05.357588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:05.418052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2fa24-8589-44b5-bd77-1fd64da48db0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.437590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017367b-7710-4850-a165-49603812600d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.441715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:05.466997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.468326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.469024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.469446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:05.470486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:05.472813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:05.476368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.476968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.477242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.477509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:05.478378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:05.478680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:05.480231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:05.481741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 459/905] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:45:05 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:45:05.510022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 447/906] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Sep 19 11:45:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:05.511882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:05.625054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40e966-d523-4b91-bb03-dd7ad869517d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.663912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.664530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.664804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.665072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:05.717197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:05.737913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:05.739342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:05.842529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 460/907] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:05 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:05.848409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:05.943136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:05.960248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df52de6-4a4e-43af-82dd-a16a5c2fd1f6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:05.996217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:05.996771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:05.997051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:05.997321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:06.009413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:06.010949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:06.026329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:06.027339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:06.028466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:06.028466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:06.028466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:06.028466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 448/908] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:06.034814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:06.123382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:06.153913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3581bf5-e402-4bb4-943a-865e5a778a02 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:06.197466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:06.198122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:06.198372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:06.198624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:06.199271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:06.199573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:06.200927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:06.213617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 461/909] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:06.219243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:06.330245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeadd78-3f4b-4d20-85e0-191efd91d440 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:06.368789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:06.369435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:06.369806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:06.370092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:06.373082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:06.380126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:06.381641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:06.394839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:06.396919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:06.399173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:06.400818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 449/910] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:06.405784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:06.516298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6c99-58ce-40cc-b46b-0e2b8cb4a626 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:06.556482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07add6c3-de6e-4f42-8108-560b907f1f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:06.557096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07add6c3-de6e-4f42-8108-560b907f1f5e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:06.557526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07add6c3-de6e-4f42-8108-560b907f1f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:06.557831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07add6c3-de6e-4f42-8108-560b907f1f5e None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:06.558538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07add6c3-de6e-4f42-8108-560b907f1f5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:06.558848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07add6c3-de6e-4f42-8108-560b907f1f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:06.560394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07add6c3-de6e-4f42-8108-560b907f1f5e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:06.583953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:06.583953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 462/911] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:06.586029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:06.667934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cfad5-6286-4981-a533-2bfb801b7b85 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:06.667934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cfad5-6286-4981-a533-2bfb801b7b85 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:06.667934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cfad5-6286-4981-a533-2bfb801b7b85 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:06.668616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 463/912] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:06.674152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b87fdf-dbe4-4001-9067-ce34ad3ba74a None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:06.674678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b87fdf-dbe4-4001-9067-ce34ad3ba74a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:06.675189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b87fdf-dbe4-4001-9067-ce34ad3ba74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:06.695652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-80b87fdf-dbe4-4001-9067-ce34ad3ba74a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:06.696067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329a4f2-4e50-485c-a664-72cce671e2bf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:06.698139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-80b87fdf-dbe4-4001-9067-ce34ad3ba74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:06.744639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:06.745632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:06.746226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:06.746716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:06.747648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] RBAC: Adding query filter params (role.id=2808f4215cc243e7a9059f529313f530, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=79ac4874ca5043ac9ef8a47f999cac97) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:06.748173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:06.750437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:06.758807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 450/913] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:06 2022] GET /identity/v3/role_assignments?user.id=79ac4874ca5043ac9ef8a47f999cac97&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=2808f4215cc243e7a9059f529313f530 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:06.783101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 464/914] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:07.052206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:07.122409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:07.168159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046938cc-a652-48ae-b02f-d934af601197 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:07.204936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60328e51-f434-43b3-818b-da162bafb33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:07.205648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60328e51-f434-43b3-818b-da162bafb33f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.205970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60328e51-f434-43b3-818b-da162bafb33f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.206241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60328e51-f434-43b3-818b-da162bafb33f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.206847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60328e51-f434-43b3-818b-da162bafb33f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:07.208335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60328e51-f434-43b3-818b-da162bafb33f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-60328e51-f434-43b3-818b-da162bafb33f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:07.219171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:07.220226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:07.221142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 451/915] 10.222.0.88 () {56 vars in 1132 bytes} [Mon Sep 19 11:45:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:07.228841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b87fdf-dbe4-4001-9067-ce34ad3ba74a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:07.319975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f2d6-4f7a-4571-8b50-e1a14f8045c9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.320603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f2d6-4f7a-4571-8b50-e1a14f8045c9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.321169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f2d6-4f7a-4571-8b50-e1a14f8045c9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.324209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 452/916] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:07.329912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71c93c-ab52-4b79-96e6-7844dcaa48e4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.330461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71c93c-ab52-4b79-96e6-7844dcaa48e4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.331361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71c93c-ab52-4b79-96e6-7844dcaa48e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.342850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b87fdf-dbe4-4001-9067-ce34ad3ba74a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:07.351106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4b71c93c-ab52-4b79-96e6-7844dcaa48e4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:07.352651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b71c93c-ab52-4b79-96e6-7844dcaa48e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:07.373554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:07.380016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:07.380626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.380901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.381168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.381815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:07.382313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:07.383811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:07.389522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 465/917] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:07.395316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:07.439552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 453/918] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:07.504095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261d8b0-2259-4e81-9b88-386f7f36a26a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:07.541240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:07.542158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.542575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.542978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.543945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:07.544404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:07.546688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:07.555000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 466/919] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:07.560881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71c93c-ab52-4b79-96e6-7844dcaa48e4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:07.624314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:07.672199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71c93c-ab52-4b79-96e6-7844dcaa48e4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:07.710204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb023-ca87-40e8-a159-c3e4e46261d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:07.711124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb023-ca87-40e8-a159-c3e4e46261d8 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.711608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb023-ca87-40e8-a159-c3e4e46261d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.712056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb023-ca87-40e8-a159-c3e4e46261d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.713428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb023-ca87-40e8-a159-c3e4e46261d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:07.715324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb023-ca87-40e8-a159-c3e4e46261d8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:07.722089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:07.740189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 454/920] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:45:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:07.810634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:07.859980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55467-480a-42d0-a92e-ec7d3f2ebdae None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:07.888765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff042-8461-415c-b4b7-11abf1f4a64c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.889533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff042-8461-415c-b4b7-11abf1f4a64c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.890792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff042-8461-415c-b4b7-11abf1f4a64c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.895149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 455/921] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:07.901225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdbdf2-9635-4880-94a8-5805b5b06611 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.901455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdbdf2-9635-4880-94a8-5805b5b06611 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.901703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdbdf2-9635-4880-94a8-5805b5b06611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.924210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:07.925155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f4cdbdf2-9635-4880-94a8-5805b5b06611 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:07.925483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:07.926033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:07.926419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f4cdbdf2-9635-4880-94a8-5805b5b06611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:07.926720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:07.927632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:07.929536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:07.940345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:07.941460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:07.942389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:07.946426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 467/922] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:07.953210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.026283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 456/923] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:08.077513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fcfc8-ee45-447c-bde1-69075a822860 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.096780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:08.115636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:08.116250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:08.116539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:08.116794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:08.117426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:08.117729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:08.119293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:08.124442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 468/924] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:08.137295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdbdf2-9635-4880-94a8-5805b5b06611 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.190198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:08.242646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdbdf2-9635-4880-94a8-5805b5b06611 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.277719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:08.278353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:08.278633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:08.278906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:08.279636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:08.279942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:08.281305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:08.286591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 457/925] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:08.292861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.301867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.410037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d45bb-49b8-478b-bba4-9fe642b6c879 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.413160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:08.426824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762539db-e027-4e02-ab93-4445ddc8fd83 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.451934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:08.452833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:08.453273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:08.454416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:08.455428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:08.457104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:08.468828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:08.469437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:08.469682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:08.470895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:45:08.471862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:08.472628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:08.472628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:08.472628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] RBAC: Adding query filter params (interface=public, service_id=92ab10c2ddb24c3bbbca436ac8b3fd57, region_id=RegionOne) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:08.472628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:08.472628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:08.474638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 458/926] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:45:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:08.478661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 469/927] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:45:08 2022] GET /identity/v3/endpoints?service_id=92ab10c2ddb24c3bbbca436ac8b3fd57&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:08.480973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.483740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.548608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:08.597342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61986312-00c5-4a53-888a-7142acbe6877 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.626361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edddccc-20c9-464f-827d-a533f178ff89 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.643605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:08.644467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:08.644747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:08.645019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:08.646062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:08.646383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:08.647764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:08.657802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 459/928] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:45:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:08.660342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.677518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:08.678738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:08.679190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:08.679623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:08.680653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:08.683901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:08.693135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 470/929] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:45:08 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:45:08.703107 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:08.789597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fccf3-72c2-4dfc-bb09-00406e405155 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.829044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5ce0d-64d0-439c-a7d1-5aeba560e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:08.829671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ce0d-64d0-439c-a7d1-5aeba560e610 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:08.830051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ce0d-64d0-439c-a7d1-5aeba560e610 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:08.830325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ce0d-64d0-439c-a7d1-5aeba560e610 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:08.830976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5ce0d-64d0-439c-a7d1-5aeba560e610 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:08.831282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5ce0d-64d0-439c-a7d1-5aeba560e610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:08.832876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5ce0d-64d0-439c-a7d1-5aeba560e610 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:08.839061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 460/930] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Sep 19 11:45:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:08.845021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:08.963174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e454a-2b29-4382-b858-42ae0717dce3 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:08.974127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:08.999922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a895eb-8750-4b8b-ac95-0441c073873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:09.000536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a895eb-8750-4b8b-ac95-0441c073873e None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:09.000823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a895eb-8750-4b8b-ac95-0441c073873e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:09.001096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a895eb-8750-4b8b-ac95-0441c073873e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:09.001790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a895eb-8750-4b8b-ac95-0441c073873e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:09.003317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a895eb-8750-4b8b-ac95-0441c073873e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:09.027421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 471/931] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:45:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:09.031381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c4a14-020f-48f8-ae22-35d0d939a84c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:09.031663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c4a14-020f-48f8-ae22-35d0d939a84c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:09.031961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c4a14-020f-48f8-ae22-35d0d939a84c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:09.033674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 461/932] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:09.039595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0ae1-99d7-4913-a59d-2b6d65da41c0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:09.039905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0ae1-99d7-4913-a59d-2b6d65da41c0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:09.040149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0ae1-99d7-4913-a59d-2b6d65da41c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:09.061148 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2abb0ae1-99d7-4913-a59d-2b6d65da41c0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:09.062432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2abb0ae1-99d7-4913-a59d-2b6d65da41c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:09.154882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 472/933] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:09.436405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c4a14-020f-48f8-ae22-35d0d939a84c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:09.540953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:09.548773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c4a14-020f-48f8-ae22-35d0d939a84c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:09.587132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:09.587800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:09.588127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:09.588405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:09.599702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:09.605310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:09.606307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:09.607749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 462/934] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Sep 19 11:45:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:09.612600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb0ae1-99d7-4913-a59d-2b6d65da41c0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:09.710226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:09.724630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb0ae1-99d7-4913-a59d-2b6d65da41c0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:09.762840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:09.763748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:09.764173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:09.764659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:09.765466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:09.765968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:09.767500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:09.777013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 473/935] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Sep 19 11:45:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:09.782369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:09.893084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff4db-9127-4ea8-8a6b-0cd9557007b6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:09.929562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:09.930219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:09.930498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:09.931337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:09.942850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:09.944289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:09.959723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:09.960800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:09.962877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:09.962877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:09.962877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:09.962877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 463/936] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:09.966999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.083920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4509a0-e812-485f-99b8-40f86e9150cc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:10.122082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54bb4f-5027-4a81-a5b6-27f7bad99266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:10.122695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54bb4f-5027-4a81-a5b6-27f7bad99266 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.122972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54bb4f-5027-4a81-a5b6-27f7bad99266 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.123241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54bb4f-5027-4a81-a5b6-27f7bad99266 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.123885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54bb4f-5027-4a81-a5b6-27f7bad99266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:10.124187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54bb4f-5027-4a81-a5b6-27f7bad99266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:10.125523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54bb4f-5027-4a81-a5b6-27f7bad99266 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:10.138743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 474/937] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:10.145982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.159958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:10.194422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dc754-d8b5-40da-9664-bc1ad75e2f62 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.194761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dc754-d8b5-40da-9664-bc1ad75e2f62 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.194983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dc754-d8b5-40da-9664-bc1ad75e2f62 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.197102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 475/938] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:10.202891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b227a3f-9746-40b7-8a5c-2a17f06f63f4 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.203175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b227a3f-9746-40b7-8a5c-2a17f06f63f4 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.203694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b227a3f-9746-40b7-8a5c-2a17f06f63f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.223962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-1b227a3f-9746-40b7-8a5c-2a17f06f63f4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:10.225070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1b227a3f-9746-40b7-8a5c-2a17f06f63f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:10.236855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:10.268835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905d907-ebe7-4814-b56f-a9cb7017fb16 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:10.317920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 476/939] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:10.326688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:10.327659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.328158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.328889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.337354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:10.339225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:45:10.350946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:10.351934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:45:10.353444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:10.356053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 464/940] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:45:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:10.361860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b227a3f-9746-40b7-8a5c-2a17f06f63f4 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.467288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b227a3f-9746-40b7-8a5c-2a17f06f63f4 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:10.504106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:10.505058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.505642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.506167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.507166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:10.507803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:10.509507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:10.528653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 477/941] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Sep 19 11:45:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:10.535156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.608370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:10.609681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.615064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:10.646713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fda113-3a25-4737-8f55-0c7447f43504 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:10.682043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:10.682591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.682870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.683142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.694393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:10.695654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:10.712747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:10.713918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:10.714923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:10.714923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:10.714923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:10.714923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 465/942] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:10.720167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.747695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49bb5a-fa7a-48d0-8386-156a26c7e3de None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:10.789382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:10.790099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.791084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.791364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.792006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:10.793487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:10.803793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:10.804904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:10.805859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 478/943] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:10.814078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.852074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa86c4-42b9-4ced-ad78-82f77e888b5b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:10.882365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:10.901497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:10.902206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.902490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.903170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:10.903331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.904349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:10.904586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:10.906326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:10.923106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 466/944] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:10.929059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:10.957892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3858a-c5ed-4b81-b299-023dba5c184c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:10.992705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b4f6a-7c34-4942-bdba-96c8a10133ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:10.993339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b4f6a-7c34-4942-bdba-96c8a10133ad None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:10.993619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b4f6a-7c34-4942-bdba-96c8a10133ad None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:10.993918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b4f6a-7c34-4942-bdba-96c8a10133ad None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:10.994543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b4f6a-7c34-4942-bdba-96c8a10133ad None admin] RBAC: Adding query filter params (name=compute) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:10.994844 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b4f6a-7c34-4942-bdba-96c8a10133ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:10.996186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b4f6a-7c34-4942-bdba-96c8a10133ad None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.001103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 479/945] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:45:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:11.002362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5625-9dfc-4d0e-8a0b-be1dec08385f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.002645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5625-9dfc-4d0e-8a0b-be1dec08385f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.002913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5625-9dfc-4d0e-8a0b-be1dec08385f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.004239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 480/946] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:11.006656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd5625-9dfc-4d0e-8a0b-be1dec08385f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:11.049578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c262902-2e7a-4616-ae8f-050b6cd84c4b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:11.089565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313b96b-474d-4d3f-96d9-f6d4881c3365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:11.090225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313b96b-474d-4d3f-96d9-f6d4881c3365 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.090501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313b96b-474d-4d3f-96d9-f6d4881c3365 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.090767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313b96b-474d-4d3f-96d9-f6d4881c3365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.098205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313b96b-474d-4d3f-96d9-f6d4881c3365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:11.099629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313b96b-474d-4d3f-96d9-f6d4881c3365 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b313b96b-474d-4d3f-96d9-f6d4881c3365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:11.111446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:11.112417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:11.113359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 467/947] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:11.114264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257ebb6-cea5-4373-9f9c-4c3640a28725 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.114264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257ebb6-cea5-4373-9f9c-4c3640a28725 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.114264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257ebb6-cea5-4373-9f9c-4c3640a28725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.126893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd5625-9dfc-4d0e-8a0b-be1dec08385f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:11.136226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2257ebb6-cea5-4373-9f9c-4c3640a28725 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:11.137828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2257ebb6-cea5-4373-9f9c-4c3640a28725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:11.165403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:11.172012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:11.172736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.173010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.173275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.173935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:11.174359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:11.175516 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:11.175718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.180715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 481/948] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:45:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:11.182057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:11.223589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 468/949] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:11.312451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3453e-c8b5-40cd-871e-cc9c5144797b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:11.352421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:11.353556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.354087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.354572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.355535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:11.356060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:11.357788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.372470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 482/950] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:45:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:11.377997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dcdb9-7677-445d-90c4-261e8c1d359a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.378541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dcdb9-7677-445d-90c4-261e8c1d359a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.378772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:11.378896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dcdb9-7677-445d-90c4-261e8c1d359a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.380404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 469/951] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:11.386434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ce15-887f-4512-9f35-8b4c6eb9ca4d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.386730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ce15-887f-4512-9f35-8b4c6eb9ca4d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.386981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ce15-887f-4512-9f35-8b4c6eb9ca4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.410733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-97c3ce15-887f-4512-9f35-8b4c6eb9ca4d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:11.411927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97c3ce15-887f-4512-9f35-8b4c6eb9ca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:11.444572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:11.499730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddbb50-4b2c-4875-b762-07309cc4ecb9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:11.508353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 470/952] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:11.514434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ce15-887f-4512-9f35-8b4c6eb9ca4d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:11.540198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:11.541720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.542206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.542659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/79ac4874ca5043ac9ef8a47f999cac97/roles/2808f4215cc243e7a9059f529313f530` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.545970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=79ac4874ca5043ac9ef8a47f999cac97, role_id=2808f4215cc243e7a9059f529313f530)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:11.548370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.568233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 483/953] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:45:11 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/79ac4874ca5043ac9ef8a47f999cac97/roles/2808f4215cc243e7a9059f529313f530 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:45:11.645680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:11.649822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ce15-887f-4512-9f35-8b4c6eb9ca4d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:11.693119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:11.693794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.694084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.694360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.694971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:11.696494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:11.707553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:11.709049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:11.710310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 471/954] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:11.719128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:11.812904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:11.843663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f482357-45ba-4676-87dd-a940143d7bc1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:11.859863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:11.885313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:11.886163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.886517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.886794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.887435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:11.887804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:11.889220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.895602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 484/955] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:11.901507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:11.947966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49295270-6495-4904-9d0e-32a73b7ffabc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:11.984386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:11.985004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:11.985290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:11.985570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:11.986274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:11.987720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:11.990382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING keystone.server.flask.common [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 11:45:12.010772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:45:12.010772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 11:45:12.010772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [parameters: {'id': 'b1fc1453aeb542e2875b282adf59bcea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 11:45:12.010772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:45:12.015983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f546ea-252a-4c8b-be51-2a187f3f27db None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1fc1453aeb542e2875b282adf59bcea.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1fc1453aeb542e2875b282adf59bcea. Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:45:12.028512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 19 11:45:12.029507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Sep 19 11:45:12.030855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:45:12.031915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:45:12.032913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:45:12.033823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'id': 'b1fc1453aeb542e2875b282adf59bcea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:12.035027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:12.036025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1fc1453aeb542e2875b282adf59bcea. Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.037211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 472/956] 10.222.0.88 () {60 vars in 1140 bytes} [Mon Sep 19 11:45:11 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:12.048110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:12.055775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:12.056394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:12.056668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:12.056939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:12.057575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:12.057910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:12.059379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:12.065440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 485/957] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:12.070856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:12.072414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:12.132724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:12.162675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5034fc-bb98-4f82-8063-2770cb7cfb6c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:12.201716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadac6d5-8dd5-447f-9a33-1dd1d1be4776 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:12.204025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:12.204934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:12.205366 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:12.205860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:12.213865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:12.215732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:12.228508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:12.229479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:12.230453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:12.232693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 473/958] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:12.238717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:12.240813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1529672-3a40-4a65-9c23-42a343b21abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:12.241427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1529672-3a40-4a65-9c23-42a343b21abf None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:12.241708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1529672-3a40-4a65-9c23-42a343b21abf None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:12.242009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1529672-3a40-4a65-9c23-42a343b21abf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:12.242791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1529672-3a40-4a65-9c23-42a343b21abf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:12.244952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1529672-3a40-4a65-9c23-42a343b21abf None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:12.273865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 486/959] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:45:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:45:12.348696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:12.373731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14956f-708e-44c0-b2d6-bfd5211f4ac4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:12.425404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2df0c-619f-4256-9a30-5c560d2a0cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:12.426431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2df0c-619f-4256-9a30-5c560d2a0cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:12.426922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2df0c-619f-4256-9a30-5c560d2a0cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:12.427377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2df0c-619f-4256-9a30-5c560d2a0cb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:12.428260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2df0c-619f-4256-9a30-5c560d2a0cb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:12.428756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2df0c-619f-4256-9a30-5c560d2a0cb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:12.430557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2df0c-619f-4256-9a30-5c560d2a0cb1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:12.453196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 474/960] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:45:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:13.458571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec4518-56af-470e-bca5-f201f5e31e23 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:13.459083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec4518-56af-470e-bca5-f201f5e31e23 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:13.459083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec4518-56af-470e-bca5-f201f5e31e23 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:13.460596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 487/961] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:13.467790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb7cf1-4388-4810-9f51-24dfbf96a7fd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:13.468207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb7cf1-4388-4810-9f51-24dfbf96a7fd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:13.468382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb7cf1-4388-4810-9f51-24dfbf96a7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:13.487230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8dbb7cf1-4388-4810-9f51-24dfbf96a7fd None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:13.488751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8dbb7cf1-4388-4810-9f51-24dfbf96a7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:13.558653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 475/962] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:13.852751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec4518-56af-470e-bca5-f201f5e31e23 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:13.871367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2078d-25f5-41fe-9713-59a7a1f030b5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:13.871708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2078d-25f5-41fe-9713-59a7a1f030b5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:13.871929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2078d-25f5-41fe-9713-59a7a1f030b5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:13.873435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 476/963] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:13.878426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3ce8f-4c66-4e46-9dd8-92663e77b093 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:13.878712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3ce8f-4c66-4e46-9dd8-92663e77b093 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:13.879122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3ce8f-4c66-4e46-9dd8-92663e77b093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:13.897461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a9a3ce8f-4c66-4e46-9dd8-92663e77b093 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:13.898682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a9a3ce8f-4c66-4e46-9dd8-92663e77b093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:13.963740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:13.978194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 477/964] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:13.983861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec4518-56af-470e-bca5-f201f5e31e23 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:13.994124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:14.018993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:14.020047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.020480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.021065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.022253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:14.023778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:14.039834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 488/965] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:45:13 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:14.261013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3ce8f-4c66-4e46-9dd8-92663e77b093 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:14.370647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3ce8f-4c66-4e46-9dd8-92663e77b093 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:14.413959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e57bd-9432-4749-9b20-21ad84e539d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:14.414515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e57bd-9432-4749-9b20-21ad84e539d5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.414793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e57bd-9432-4749-9b20-21ad84e539d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.415176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e57bd-9432-4749-9b20-21ad84e539d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.427173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e57bd-9432-4749-9b20-21ad84e539d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ae5e57bd-9432-4749-9b20-21ad84e539d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:14.432839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:14.433833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:14.434723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 478/966] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Sep 19 11:45:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:14.440214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:14.556576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd0990-26c1-448a-9e1f-9bd013430487 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:14.594744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:14.595365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.595646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.595921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.596599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:14.596775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65795c7b-97e1-4244-8ffe-2afff5942f17 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.597286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:14.597375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65795c7b-97e1-4244-8ffe-2afff5942f17 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.597375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65795c7b-97e1-4244-8ffe-2afff5942f17 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.598283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:14.598868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 479/967] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:14.604421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfc3a0-a353-430f-99f8-496c4b149943 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.604706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfc3a0-a353-430f-99f8-496c4b149943 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.604978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfc3a0-a353-430f-99f8-496c4b149943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.607869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 489/968] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Sep 19 11:45:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:14.613562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:14.629338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-48bfc3a0-a353-430f-99f8-496c4b149943 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:14.630997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-48bfc3a0-a353-430f-99f8-496c4b149943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:14.650096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:14.687722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:14.718148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 480/969] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:14.741940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0429306-9d0d-4e81-b18e-e3ae1b984a09 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:14.779500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ed0e9-61b4-459d-9a6b-e90cf76b5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:14.780326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed0e9-61b4-459d-9a6b-e90cf76b5c1d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.780602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed0e9-61b4-459d-9a6b-e90cf76b5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.780869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed0e9-61b4-459d-9a6b-e90cf76b5c1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.791206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ed0e9-61b4-459d-9a6b-e90cf76b5c1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:14.792634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ed0e9-61b4-459d-9a6b-e90cf76b5c1d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-741ed0e9-61b4-459d-9a6b-e90cf76b5c1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:14.807922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:14.808952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:14.809945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:14.809945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:14.809945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:14.809945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 490/970] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:14.816626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfc3a0-a353-430f-99f8-496c4b149943 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:14.829567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32480dee-ec1d-4407-a3ff-07133ef765ca None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.829894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32480dee-ec1d-4407-a3ff-07133ef765ca None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.830151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32480dee-ec1d-4407-a3ff-07133ef765ca None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.831747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 491/971] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:14.836741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaec69-c777-448e-be2b-465eab67adf8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.837623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaec69-c777-448e-be2b-465eab67adf8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.837940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaec69-c777-448e-be2b-465eab67adf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.859499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-6daaec69-c777-448e-be2b-465eab67adf8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:14.860699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6daaec69-c777-448e-be2b-465eab67adf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:14.923330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfc3a0-a353-430f-99f8-496c4b149943 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:14.947666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 492/972] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:14.966258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:14.967286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.967774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.968261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.969116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:14.969622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:14.971286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:14.984044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 481/973] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:14.990122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaec69-c777-448e-be2b-465eab67adf8 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:14.991401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:14.993880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.050168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.119172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ab2fe-98d8-4963-85ad-d6b53796fb8c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.128134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaec69-c777-448e-be2b-465eab67adf8 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.160752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.161821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.162092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.162851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.163528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.165554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.175323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.176217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.176679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:15.177208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:15.178257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:15.179180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.181254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.183851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 482/974] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:15.185977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.190144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.192802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:45:15.210343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:15.211388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:45:15.212904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:15.220676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 493/975] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:45:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:15.221943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.291897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.312400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef62379-cdfa-47a9-b1b2-81d987401701 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.321003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.335511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04536a-7c13-4b68-b7cd-ef1abb864d3a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.349160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.349796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.350082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.350359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.350985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:15.351294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.353390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.360410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 483/976] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:15.361221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.383282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.384431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.384738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.384988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.385720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:15.386060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.387478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.405789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 494/977] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:15.406629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.487832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33a2e7-21ca-408a-a2b5-422f05bcb4a4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.520790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a7bfe-43e6-4f8b-bc87-8b05d057d6bf None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.527615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.528437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.529000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.529287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.529959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.531396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.543184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.546933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:45:15.546933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 11:45:15.546933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [parameters: {'user_id': '29524c9d7a2a41d698c4501216490657', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 11:45:15.546933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 19 11:45:15.553925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:45:15.554830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:45:15.555910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:45:15.556829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:45:15.558644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:45:15.559749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'user_id': '29524c9d7a2a41d698c4501216490657', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:15.560646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 19 11:45:15.561480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 484/978] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Sep 19 11:45:15 2022] POST /identity/v3/users => generated 160 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:15.564741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.569250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.570281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.570635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.571066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.572248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:15.572583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.574538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.586213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 495/979] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:15.586213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.596721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.673446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466bc95-cb0c-4ab2-9d2e-7a4085bce3af None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.691057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd63e-ae87-4e35-bc92-57187e95a6b7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.711487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.712622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.712897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.713161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.730222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.731015 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.731248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.731468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.739177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.740531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:45:15.753338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:15.754592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:15.755573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 496/980] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:15.756222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.793548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.795394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.806302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.831720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:15.876593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44678c27-d2c9-4895-8998-06446e08afd3 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:15.908124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 485/981] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:15.908992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:15.924171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:15.924950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:15.925232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:15.925508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:15.926260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] RBAC: Adding query filter params (interface=admin, service_id=92ab10c2ddb24c3bbbca436ac8b3fd57, region_id=RegionOne) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:15.926573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:15.927960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:15.933362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 497/982] 10.222.0.88 () {56 vars in 1249 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/endpoints?service_id=92ab10c2ddb24c3bbbca436ac8b3fd57&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:15.934187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:16.022057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0ab07-7d23-49ff-8d2f-1805717b93e9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.041672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f0319-9395-4d23-ba42-5d19fb052baa None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.050969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:16.053607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:16.060259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.060879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.061159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.061425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.062158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:16.062461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:16.063905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:16.080269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.080935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.081218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.081495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.093386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:16.094978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:16.103945 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 486/983] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:16.106717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:16.111601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:16.112723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:16.113665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:16.113665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:16.113665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:16.113665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 498/984] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:16.120488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:16.238114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f66ad-ce4b-46e9-ae76-a5f20e39b6eb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.277944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.278544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.278758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.279182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.279809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:16.281268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:16.287334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7da87-cc99-476a-9749-6a605c334e73 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.295013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 487/985] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:45:16 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:45:16.333158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff62f1-e248-4f88-ba3a-f32efd9083fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.333827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff62f1-e248-4f88-ba3a-f32efd9083fd None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.334111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff62f1-e248-4f88-ba3a-f32efd9083fd None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.334383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff62f1-e248-4f88-ba3a-f32efd9083fd None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.335058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff62f1-e248-4f88-ba3a-f32efd9083fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:16.335366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff62f1-e248-4f88-ba3a-f32efd9083fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:16.336781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff62f1-e248-4f88-ba3a-f32efd9083fd None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:16.354493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 499/986] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:16.361028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:16.433842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:16.445998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42cc3-25dc-4f23-bb30-a5974ea3bd8f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.446407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42cc3-25dc-4f23-bb30-a5974ea3bd8f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.446599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42cc3-25dc-4f23-bb30-a5974ea3bd8f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.448107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 500/987] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:16.454679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12527e-ce66-429c-b5f4-41a14df4ceeb None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.454963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12527e-ce66-429c-b5f4-41a14df4ceeb None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.455241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12527e-ce66-429c-b5f4-41a14df4ceeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.475885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-be12527e-ce66-429c-b5f4-41a14df4ceeb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:16.477260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-be12527e-ce66-429c-b5f4-41a14df4ceeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:16.494287 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:16.497677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147f9cf-6313-4fed-a2f3-f6725194ec2d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.532492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.533041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.533314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.533578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.540324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:16.541740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:16.554022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:16.555075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:16.556570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 488/988] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:16.561436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:16.563009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 501/989] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:16.667443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778a53a-978d-48cf-a107-0811d1e3f7ca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.681520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:16.704546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.705153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.705434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.705705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.706420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:16.706725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:16.708140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:16.728055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 489/990] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:16.729795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12527e-ce66-429c-b5f4-41a14df4ceeb None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:16.835020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:16.840020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12527e-ce66-429c-b5f4-41a14df4ceeb None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.876617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.877225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.877513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.877808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.878547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] RBAC: Adding query filter params (role.id=2808f4215cc243e7a9059f529313f530, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=79ac4874ca5043ac9ef8a47f999cac97) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:16.878861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:16.880281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:16.889953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 502/991] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:16 2022] GET /identity/v3/role_assignments?user.id=79ac4874ca5043ac9ef8a47f999cac97&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=2808f4215cc243e7a9059f529313f530 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:16.959870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f6f9e-f7cc-4e10-b88c-34050436d015 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:16.997786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:16.998453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:16.998727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:16.998996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:16.999584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:17.000950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:17.010956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:17.012095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:17.012978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 490/992] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:17.018311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:17.042147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:17.140814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412309f-2505-4c62-a70e-5c6ec3b83fe9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:17.179499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:17.180110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:17.180401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:17.180805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:17.181443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:17.181749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:17.183128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:17.188262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 503/993] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:45:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:17.193943 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:17.239912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:17.304132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d817-f5de-47f7-82f5-fa85935944a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:17.338498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:17.339104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:17.339424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:17.339656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:17.340285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:17.340590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:17.341969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:17.352015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 491/994] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:45:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:17.353556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:17.436012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:17.458953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269679fd-3b01-43d0-9a4a-50ff9887c2b9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:17.494323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdcd95-3f40-4bbe-bc69-7b9410c45b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:17.494926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdcd95-3f40-4bbe-bc69-7b9410c45b23 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:17.495201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdcd95-3f40-4bbe-bc69-7b9410c45b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:17.495471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdcd95-3f40-4bbe-bc69-7b9410c45b23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:17.496166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdcd95-3f40-4bbe-bc69-7b9410c45b23 None admin] RBAC: Adding query filter params (interface=public, service_id=11b9209c0c964520b516171279881262, region_id=RegionOne) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:17.496470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdcd95-3f40-4bbe-bc69-7b9410c45b23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:17.498419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdcd95-3f40-4bbe-bc69-7b9410c45b23 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:17.508711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 504/995] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:45:17 2022] GET /identity/v3/endpoints?service_id=11b9209c0c964520b516171279881262&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:17.509926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:17.613344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46a860-8b1e-4321-ba9f-53233724419a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:17.614941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:17.650550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc45ba6-bfb0-49cf-915c-90fe9d73eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:17.651256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45ba6-bfb0-49cf-915c-90fe9d73eca9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:17.651454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45ba6-bfb0-49cf-915c-90fe9d73eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:17.651710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45ba6-bfb0-49cf-915c-90fe9d73eca9 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:17.652319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc45ba6-bfb0-49cf-915c-90fe9d73eca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:17.653680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc45ba6-bfb0-49cf-915c-90fe9d73eca9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:17.665813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 492/996] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:45:17 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:45:18.505533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a982c6-8a89-49a7-b29b-671c795bad19 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:18.506068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a982c6-8a89-49a7-b29b-671c795bad19 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:18.506151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a982c6-8a89-49a7-b29b-671c795bad19 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:18.507476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 505/997] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:18.513424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a23e79-db7e-4268-a72a-d73c72eaffbb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:18.513869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a23e79-db7e-4268-a72a-d73c72eaffbb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:18.513960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a23e79-db7e-4268-a72a-d73c72eaffbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:18.533174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-22a23e79-db7e-4268-a72a-d73c72eaffbb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:18.534842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22a23e79-db7e-4268-a72a-d73c72eaffbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:18.605797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 493/998] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:18.881152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a982c6-8a89-49a7-b29b-671c795bad19 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:18.992434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a982c6-8a89-49a7-b29b-671c795bad19 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:19.028288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b78fa-1d86-49d1-9237-1896126be3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:19.028915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b78fa-1d86-49d1-9237-1896126be3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.029195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b78fa-1d86-49d1-9237-1896126be3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.029465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b78fa-1d86-49d1-9237-1896126be3f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.030142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b78fa-1d86-49d1-9237-1896126be3f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:19.031518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b78fa-1d86-49d1-9237-1896126be3f4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-103b78fa-1d86-49d1-9237-1896126be3f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:19.040423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:19.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:19.042401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 506/999] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Sep 19 11:45:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:19.047432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a23e79-db7e-4268-a72a-d73c72eaffbb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:19.093185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ed0fc-3445-4338-abe9-7676faba0b6f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.093465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ed0fc-3445-4338-abe9-7676faba0b6f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.093738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ed0fc-3445-4338-abe9-7676faba0b6f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.095219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 507/1000] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:19.100042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c81d75-27b8-48e1-ab8d-889beffec2b6 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.100333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c81d75-27b8-48e1-ab8d-889beffec2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.100610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c81d75-27b8-48e1-ab8d-889beffec2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.113910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:19.119939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-f3c81d75-27b8-48e1-ab8d-889beffec2b6 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:19.121135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f3c81d75-27b8-48e1-ab8d-889beffec2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:19.155852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:19.164119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a23e79-db7e-4268-a72a-d73c72eaffbb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:19.200826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 508/1001] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:19.201718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:19.202356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.202633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.202902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.203785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:19.204086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:19.205433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:19.210592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 494/1002] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:19.216835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c81d75-27b8-48e1-ab8d-889beffec2b6 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:19.317853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c81d75-27b8-48e1-ab8d-889beffec2b6 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:19.352749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:19.353358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.353639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.353934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.354563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:19.354867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:19.356212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:19.358448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:19.365937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 509/1003] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Sep 19 11:45:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:19.369335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:19.468343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:19.475872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56ca45-edba-4242-8f70-2a3d33991b91 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:19.512285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:19.512934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.513214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.513488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.514214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:19.515585 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:19.545787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 495/1004] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:45:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:19.591818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ba55f-8873-481a-b940-e1478d3efc2b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:19.637319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:19.637996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.638275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.638545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] PATH_INFO: `/v3/services/image` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.639133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:19.640495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:19.650101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:19.651182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:19.652045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 510/1005] 10.222.0.88 () {56 vars in 1105 bytes} [Mon Sep 19 11:45:19 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:19.657564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:19.683637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:19.779245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0f221-7e53-45c5-a051-f5ec55ee3479 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:19.814115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc321-5836-413f-9dfa-62a2a8e83310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:19.814741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc321-5836-413f-9dfa-62a2a8e83310 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.815023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc321-5836-413f-9dfa-62a2a8e83310 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.815299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc321-5836-413f-9dfa-62a2a8e83310 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.815939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbc321-5836-413f-9dfa-62a2a8e83310 None admin] RBAC: Adding query filter params (name=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:19.816249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbc321-5836-413f-9dfa-62a2a8e83310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:19.818515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbc321-5836-413f-9dfa-62a2a8e83310 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:19.830057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 496/1006] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:19.833144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:19.879998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356af51-5230-4af1-82e0-0125c4835544 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.880269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356af51-5230-4af1-82e0-0125c4835544 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.880545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356af51-5230-4af1-82e0-0125c4835544 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.881924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 497/1007] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:19.886987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08035-1c01-42dd-92a5-562be183ee76 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.887277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08035-1c01-42dd-92a5-562be183ee76 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.887553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08035-1c01-42dd-92a5-562be183ee76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.899348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:19.906227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4ae08035-1c01-42dd-92a5-562be183ee76 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:19.907513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ae08035-1c01-42dd-92a5-562be183ee76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:19.946142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4335e85-3b8a-44e9-955b-c4ed36bb5b92 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:19.977991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 498/1008] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:19.985656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:19.986312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:19.986579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:19.986848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:19.987477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] RBAC: Adding query filter params (type=image) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:19.987778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:19.989124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:20.001879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 511/1009] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:20.256261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08035-1c01-42dd-92a5-562be183ee76 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:20.272075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:20.376261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08035-1c01-42dd-92a5-562be183ee76 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:20.411391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:20.411940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:20.412215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:20.412491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:20.413068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:20.414441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:20.423508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:20.424569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:45:20.425457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:20.429539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 499/1010] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:20.431095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:20.539416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53b0be-958f-4c30-983f-00be1be21513 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:20.563930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:20.575686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:20.576291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:20.576571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:20.576845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:20.577470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:20.577794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:20.579392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:20.584083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 512/1011] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:45:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:20.589884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:20.671475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:20.693818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a9a4f-64ba-4b47-9bd4-c81e1f28918f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:20.729037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbc6d4-73c2-44bd-a290-8ba61784d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:20.729642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc6d4-73c2-44bd-a290-8ba61784d64b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:20.729962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc6d4-73c2-44bd-a290-8ba61784d64b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:20.730236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc6d4-73c2-44bd-a290-8ba61784d64b None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:20.730872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbc6d4-73c2-44bd-a290-8ba61784d64b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:20.731175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbc6d4-73c2-44bd-a290-8ba61784d64b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:20.732736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbc6d4-73c2-44bd-a290-8ba61784d64b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:20.742536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 500/1012] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Sep 19 11:45:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:20.744659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:20.848826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24efba-3efa-40ac-b773-557e5447c720 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:20.884217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd31c90-4f53-44c0-80e8-34be16785dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:20.884841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31c90-4f53-44c0-80e8-34be16785dfc None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:20.885116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31c90-4f53-44c0-80e8-34be16785dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:20.885385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31c90-4f53-44c0-80e8-34be16785dfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:20.886105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd31c90-4f53-44c0-80e8-34be16785dfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:20.887495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd31c90-4f53-44c0-80e8-34be16785dfc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:20.917790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 513/1013] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:45:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:21.794605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afe934-a3ca-4736-adaa-d3b656a475c6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:21.795135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afe934-a3ca-4736-adaa-d3b656a475c6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:21.795135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afe934-a3ca-4736-adaa-d3b656a475c6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:21.796847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 501/1014] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:21.801993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220454-d093-474d-8cb1-d5b20ab13604 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:21.802444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220454-d093-474d-8cb1-d5b20ab13604 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:21.802625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220454-d093-474d-8cb1-d5b20ab13604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:21.821486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ce220454-d093-474d-8cb1-d5b20ab13604 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:21.822717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ce220454-d093-474d-8cb1-d5b20ab13604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:21.868850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:21.914005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 514/1015] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:21 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:22.185665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354d0f1-b5e7-45d9-b667-be02f13e684a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:22.186045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354d0f1-b5e7-45d9-b667-be02f13e684a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:22.186343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354d0f1-b5e7-45d9-b667-be02f13e684a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:22.187837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce220454-d093-474d-8cb1-d5b20ab13604 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:22.188000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 502/1016] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:22.193292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf5166-3985-44ee-8343-4246708522a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:22.193574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf5166-3985-44ee-8343-4246708522a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:22.193894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf5166-3985-44ee-8343-4246708522a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:22.217529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-40bf5166-3985-44ee-8343-4246708522a0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:22.218771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40bf5166-3985-44ee-8343-4246708522a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:22.297793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 503/1017] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:22 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:22.313909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce220454-d093-474d-8cb1-d5b20ab13604 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:22.348686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:22.349311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:22.349589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:22.349887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:22.350976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:22.352912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:22.358118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 515/1018] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:22.360264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf5166-3985-44ee-8343-4246708522a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:22.431999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:22.463443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf5166-3985-44ee-8343-4246708522a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:22.498211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc837-e8d8-4e46-880f-a9d34792c0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:22.498811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc837-e8d8-4e46-880f-a9d34792c0b1 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:22.499085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc837-e8d8-4e46-880f-a9d34792c0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:22.499352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc837-e8d8-4e46-880f-a9d34792c0b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:22.500003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc837-e8d8-4e46-880f-a9d34792c0b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:22.501409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc837-e8d8-4e46-880f-a9d34792c0b1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:22.529098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 504/1019] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:45:22 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:22.568935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:22.616879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:22.686332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963c50d-e13b-4fe9-9513-e4f7693c9adf None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:22.724842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f69d32-417e-482d-b675-a7701d5e4ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:22.725450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f69d32-417e-482d-b675-a7701d5e4ca3 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:22.725729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f69d32-417e-482d-b675-a7701d5e4ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:22.726019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f69d32-417e-482d-b675-a7701d5e4ca3 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:22.726687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f69d32-417e-482d-b675-a7701d5e4ca3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:22.728118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f69d32-417e-482d-b675-a7701d5e4ca3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:22.746924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 516/1020] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:45:22 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:23.254420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e7b9-364c-4852-aabb-89bdca2c5606 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:23.254929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e7b9-364c-4852-aabb-89bdca2c5606 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:23.255008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e7b9-364c-4852-aabb-89bdca2c5606 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:23.256451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 505/1021] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:23.261800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed946-a48a-43f3-ac2f-afc920251d50 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:23.262198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed946-a48a-43f3-ac2f-afc920251d50 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:23.262377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed946-a48a-43f3-ac2f-afc920251d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:23.280920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b83ed946-a48a-43f3-ac2f-afc920251d50 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:23.282137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b83ed946-a48a-43f3-ac2f-afc920251d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:23.358811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 517/1022] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:23.634561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e7b9-364c-4852-aabb-89bdca2c5606 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:23.744992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e7b9-364c-4852-aabb-89bdca2c5606 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:23.769143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:23.783183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:23.783822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:23.784104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:23.784447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:23.795403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:23.800204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:23.801384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:23.802645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:23.806092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 506/1023] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Sep 19 11:45:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:23.807593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed946-a48a-43f3-ac2f-afc920251d50 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:23.833751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:23.926945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed946-a48a-43f3-ac2f-afc920251d50 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:23.963066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:23.964032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:23.964524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:23.965005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:23.965911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:23.966415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:23.968044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:23.976599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 518/1024] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Sep 19 11:45:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:23.982184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:24.085296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffda9be-5d78-4f8b-92ad-88e7d08d435c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:24.120639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:24.121314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.121593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.121938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.133965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:24.135405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:24.154180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:24.155193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:24.156171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:24.156171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:24.156171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:24.158371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 507/1025] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:24.163928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:24.251823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:24.268988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d817-d0e6-4344-8785-8d1916561a41 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:24.303350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:24.303957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.304236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.304509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.305148 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:24.305502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:24.306868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:24.323955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 519/1026] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:24.325405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:24.364644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:24.432897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9faab6-91b5-421b-a8e2-2c29dbc7f4dd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:24.467290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:24.468160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.468609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.469023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.476455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:24.478601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:45:24.490688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:24.491795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:24.492841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 508/1027] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:45:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:24.499605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:24.608545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c1a33-ef37-4894-9649-ba8543d22b39 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:24.643411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465be91f-326e-46af-abbc-22b6989d91a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:24.644367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465be91f-326e-46af-abbc-22b6989d91a9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.644823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465be91f-326e-46af-abbc-22b6989d91a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.645266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465be91f-326e-46af-abbc-22b6989d91a9 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.646183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465be91f-326e-46af-abbc-22b6989d91a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:24.646667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465be91f-326e-46af-abbc-22b6989d91a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:24.648260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465be91f-326e-46af-abbc-22b6989d91a9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:24.672142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 520/1028] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:45:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:24.673417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:24.776484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:24.783468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e99cd-f8cd-4eb2-ad3a-17f2a0aed05d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:24.814307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83921d28-2091-4774-9e0e-f1cd39257f64 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.814594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83921d28-2091-4774-9e0e-f1cd39257f64 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.814874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83921d28-2091-4774-9e0e-f1cd39257f64 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.816203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 521/1029] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:24.819214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:24.819819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.820096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.820360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.821478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b53352-fc9f-489d-aa09-a48ac2b06a65 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.821868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b53352-fc9f-489d-aa09-a48ac2b06a65 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.822236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b53352-fc9f-489d-aa09-a48ac2b06a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.829793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:24.841816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-58b53352-fc9f-489d-aa09-a48ac2b06a65 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:24.843031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-58b53352-fc9f-489d-aa09-a48ac2b06a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:24.907122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:24.908488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:24.924887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 522/1030] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:24.981304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb2790-3e8a-4c1a-8ab0-7aa977764178 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.981647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb2790-3e8a-4c1a-8ab0-7aa977764178 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.981901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb2790-3e8a-4c1a-8ab0-7aa977764178 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:24.983230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 523/1031] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:24.988192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cc063-ab16-4d9a-851f-736ac584a8f0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:24.988478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cc063-ab16-4d9a-851f-736ac584a8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:24.988751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cc063-ab16-4d9a-851f-736ac584a8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.006380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3d2cc063-ab16-4d9a-851f-736ac584a8f0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:25.007569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3d2cc063-ab16-4d9a-851f-736ac584a8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:25.016889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:25.026157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 509/1032] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:25.035723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.094453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 524/1033] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:25.148287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44dfa-434b-4db6-ab42-867215b52d33 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.185311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.185887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.186168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.186438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.194339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cc063-ab16-4d9a-851f-736ac584a8f0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.197053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.199478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:25.215896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:25.216924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:25.217914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:25.217914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:25.217914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:25.217914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 510/1034] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:25.222700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.264078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:25.276719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:25.307051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cc063-ab16-4d9a-851f-736ac584a8f0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.334726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ac0f-ad6e-4594-aa4e-4dcd24c50dca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.343213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.343844 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.344119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.344389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.345484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.346944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.348610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 525/1035] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:25.354774 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.378360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.378969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.379246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.379520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.380161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:25.380465 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.381853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.393512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 511/1036] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:25.394191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.458955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884b045-cc2d-4fe6-bebe-1d11c9fd2303 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.487227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:25.492761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.493362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.493641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.493739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af00ec-327a-43bd-b0fd-d614b51c9e4c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.493942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.494581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.495978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.505466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:25.518606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 526/1037] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:45:25 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:45:25.519432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.538780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.539458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.540273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.540543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.541152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.542618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:25.552615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:25.553801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:25.555047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 512/1038] 10.222.0.88 () {56 vars in 1105 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:25.562861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.665436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e7f9c-5133-456e-8ea3-67df7e1304a7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.673922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04e3f8-2346-4906-a9aa-187ecd03ced7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.703176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.704107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.704390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.704667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.713265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.714325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.715289 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.715415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.715415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.715572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.717038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] RBAC: Adding query filter params (name=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:25.717038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.719509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.724663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 513/1039] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:25.726801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:25.728334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:25.729335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 527/1040] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:25.731325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.735102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.770835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:25.796586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:25.850241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03fecc-bb71-4d3b-a365-2ad56f950623 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.853461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636c923-a4c0-4d30-a441-d8f28d10f9cf None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:25.886030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.886900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.887177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.887446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.888073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:25.888371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.889716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.890201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:25.890804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:25.891107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:25.891357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:25.892050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:25.892359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:25.893790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:25.894995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 514/1041] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:25.900722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:25.906962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 528/1042] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:25.912877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:26.002428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:26.006235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9a26a-dcf0-4fd8-ba31-70d1436c2ad2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:26.027326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb87d8-63e0-4913-b8b3-d92ba9588771 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:26.037989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:26.043382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:26.043988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:26.044320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:26.044569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:26.045268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] RBAC: Adding query filter params (interface=public, service_id=ec7555c27a294b6ea826684e4485a54a, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:26.045574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:26.046958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:26.052265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 515/1043] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/endpoints?service_id=ec7555c27a294b6ea826684e4485a54a&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:26.057976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:26.067612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bce8f-d7ac-4bb3-a49a-69dd274e2400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:26.068224 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bce8f-d7ac-4bb3-a49a-69dd274e2400 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:26.068613 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bce8f-d7ac-4bb3-a49a-69dd274e2400 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:26.068812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bce8f-d7ac-4bb3-a49a-69dd274e2400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:26.069619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bce8f-d7ac-4bb3-a49a-69dd274e2400 None admin] RBAC: Adding query filter params (role.id=2808f4215cc243e7a9059f529313f530, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=345c823f8d85402ba3b07f39ed5e2fca) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:26.069961 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bce8f-d7ac-4bb3-a49a-69dd274e2400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:26.071912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bce8f-d7ac-4bb3-a49a-69dd274e2400 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:26.078991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 529/1044] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:25 2022] GET /identity/v3/role_assignments?user.id=345c823f8d85402ba3b07f39ed5e2fca&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=2808f4215cc243e7a9059f529313f530 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:26.188725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dda29-6842-4d77-a4b2-c8a44903a4c5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:26.226741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f29939-6839-4fa9-9e10-d31ed959618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:26.227455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f29939-6839-4fa9-9e10-d31ed959618b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:26.227734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f29939-6839-4fa9-9e10-d31ed959618b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:26.228006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f29939-6839-4fa9-9e10-d31ed959618b None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:26.228626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f29939-6839-4fa9-9e10-d31ed959618b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:26.230077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f29939-6839-4fa9-9e10-d31ed959618b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:26.237472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 516/1045] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:45:26 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:45:27.785523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7ad26-78bb-4cec-936d-fe05ca02d466 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:27.786092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7ad26-78bb-4cec-936d-fe05ca02d466 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:27.786092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7ad26-78bb-4cec-936d-fe05ca02d466 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:27.787572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 530/1046] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:27.792913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49402-d0a9-43fe-a2dc-942be68a1069 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:27.793372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49402-d0a9-43fe-a2dc-942be68a1069 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:27.793461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49402-d0a9-43fe-a2dc-942be68a1069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:27.812254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3ea49402-d0a9-43fe-a2dc-942be68a1069 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:27.813445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ea49402-d0a9-43fe-a2dc-942be68a1069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:27.838034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:27.884713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 517/1047] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:28.157900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7ad26-78bb-4cec-936d-fe05ca02d466 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:28.237845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe5b58-b3fe-4f47-a592-16f93531ab1b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.238135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe5b58-b3fe-4f47-a592-16f93531ab1b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.238414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe5b58-b3fe-4f47-a592-16f93531ab1b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.239754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 518/1048] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:28.244630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b88f6f-2a1b-468e-8776-aa03d5edcc31 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.245091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b88f6f-2a1b-468e-8776-aa03d5edcc31 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.245568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b88f6f-2a1b-468e-8776-aa03d5edcc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.262752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7ad26-78bb-4cec-936d-fe05ca02d466 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:28.264448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-37b88f6f-2a1b-468e-8776-aa03d5edcc31 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:28.265645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-37b88f6f-2a1b-468e-8776-aa03d5edcc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:28.300605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:28.301665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.301973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.302269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.315807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:28.321357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:28.322528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:28.329292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 531/1049] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:28.330118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:28.340125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 519/1050] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:28.365497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:28.428600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a769-8acd-4b26-8812-a231bdb0f20a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.429125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a769-8acd-4b26-8812-a231bdb0f20a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.429410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a769-8acd-4b26-8812-a231bdb0f20a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.431424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 520/1051] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:28.436457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f25d6-1496-4273-91f8-c2551f2605d5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.436766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f25d6-1496-4273-91f8-c2551f2605d5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.437011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f25d6-1496-4273-91f8-c2551f2605d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.438951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391c8f7-6f6a-425b-b1c3-b3aca1725ea4 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:28.457358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cc4f25d6-1496-4273-91f8-c2551f2605d5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:28.458556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cc4f25d6-1496-4273-91f8-c2551f2605d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:28.476403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:28.477001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.477279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.477543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.478233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:28.478533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:28.479869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:28.488538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 532/1052] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:28.494991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:28.534226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 521/1053] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:28.592925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:28.600948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f725-5223-4840-968e-5d805b0437f0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:28.604914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f25d6-1496-4273-91f8-c2551f2605d5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:28.626825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:28.638133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:28.638771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.639049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.639326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] PATH_INFO: `/v3/users/791e990074424a5098dfe26384484f67` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.653083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=791e990074424a5098dfe26384484f67)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:28.654566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:28.656236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 533/1054] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/users/791e990074424a5098dfe26384484f67 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:28.662002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:28.724100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f25d6-1496-4273-91f8-c2551f2605d5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:28.762281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:28.763307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.763582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.763912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.774200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:28.775358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161d2d0-809e-41bf-90b2-b797d9372bd5 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:28.778881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:28.779879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:28.781355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 522/1055] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:28.785843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:28.815635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:28.816249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.816502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.816774 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.838184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:28.864510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:28.884060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:28.885492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:28.886447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0088-cab8-4998-b461-b7d053350f39 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:28.927856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:28.928458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:28.928738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:28.929002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:28.929664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:28.929996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:28.931950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:28.940154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 523/1056] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:28.940944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:28.989529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 534/1057] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:45:28.990312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.040937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.047368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b988ee-b01f-4c6a-89e3-8ea037aae45d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.080671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.081288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.081560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.081853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] PATH_INFO: `/v3/services/image` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.082431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.083773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.088630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e86067-23c7-486f-8cfb-484b77d1423c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:29.092335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:29.093401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:29.094317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 524/1058] 10.222.0.88 () {56 vars in 1105 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:29.095206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.102557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.125006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.125624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.125921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.126197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.136173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.137589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:29.151929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:29.158057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:29.159229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:29.159229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:29.159229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:29.159229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 535/1059] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:29.159229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.204174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddb39f-00ca-4d97-af51-e82254729ecb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.240622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.241171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.241451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.241720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.242854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] RBAC: Authorizing `identity:get_project(project_id=aae1b727e48f4bd09aa47f65e2265b30)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.244273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.246071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 525/1060] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:29.246827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.253177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb212614-a0a0-4066-8263-086bb5326203 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.255466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.287421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.288021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.288299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.288570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.289208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] RBAC: Adding query filter params (name=image) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:29.290118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.291558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.296414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 536/1061] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:29.297197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.323163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.374856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91ce0e-3f46-4494-a84c-e59989195058 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.400656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd489740-6d9d-440b-952b-261376da5c16 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.408887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.409493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.409797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.410071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.410710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:29.411013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.412365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.424364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 526/1062] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:29.425440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.435012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.435616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.436165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.436165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.436861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=aae1b727e48f4bd09aa47f65e2265b30, user.id=791e990074424a5098dfe26384484f67) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:29.437168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.438593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.447232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 537/1063] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/role_assignments?user.id=791e990074424a5098dfe26384484f67&scope.project.id=aae1b727e48f4bd09aa47f65e2265b30&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:29.447232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.483208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.544206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fe763-8fb0-4df0-8161-dfe5e0e0e217 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.568839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c79868-ee88-4e9d-9a10-9b1620fb577d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.582131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.589952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.590594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.591173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.591524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.592238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] RBAC: Adding query filter params (type=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:29.592559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.594610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.599964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 527/1064] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:29.605739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.613402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.614075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.614355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.614628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.623570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.625011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:45:29.638070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:29.639268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:29.640390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 538/1065] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:29.645181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.751600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b3a88-504f-49b3-9f94-533b0346438d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.766110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.777357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f0400-3a14-46d9-aa7f-79ccbe957399 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:29.789707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.790457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.790898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.791174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.791951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.793306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.812497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:29.813041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:29.813319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:29.813594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:29.814370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:29.814676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:29.825840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 528/1066] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:45:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:45:29.826054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:29.846993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 539/1067] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:29.853451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:29.915943 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:29.981227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630e57-46dc-4943-9cb7-5935e2a64cd2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:30.018314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:30.019332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:30.019607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:30.019876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:30.045115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:30.046621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:30.063572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:30.064710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:30.065660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:30.065660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:30.065660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:30.069831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 529/1068] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:30.073049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:30.185301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3468b87-0686-4fe0-948c-8b273f4524f0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:30.221584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835483b6-ca74-4519-ad33-614088966513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:30.222227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835483b6-ca74-4519-ad33-614088966513 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:30.222506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835483b6-ca74-4519-ad33-614088966513 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:30.222780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835483b6-ca74-4519-ad33-614088966513 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:30.223426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835483b6-ca74-4519-ad33-614088966513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:30.223732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835483b6-ca74-4519-ad33-614088966513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:30.225084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835483b6-ca74-4519-ad33-614088966513 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:30.242509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 540/1069] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:30.243445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:30.354400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:30.359233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606e87b-ba65-4a41-bae2-08f0229a6c0f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:30.396685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:30.397314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:30.397653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:30.397964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:30.405339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:30.406819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:30.419126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:30.420176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:30.421122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 530/1070] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:30.427980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835483b6-ca74-4519-ad33-614088966513 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:30.441740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:30.537375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835483b6-ca74-4519-ad33-614088966513 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:30.571707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5597da-5c36-4c3b-91f9-d1beb13d5078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:30.572318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5597da-5c36-4c3b-91f9-d1beb13d5078 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:30.572593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5597da-5c36-4c3b-91f9-d1beb13d5078 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:30.572858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5597da-5c36-4c3b-91f9-d1beb13d5078 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:30.573553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5597da-5c36-4c3b-91f9-d1beb13d5078 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:30.573886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5597da-5c36-4c3b-91f9-d1beb13d5078 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:30.575307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5597da-5c36-4c3b-91f9-d1beb13d5078 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:30.589315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 541/1071] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:30.595055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:30.698324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726e180-9f69-4b65-8927-7505c2b96795 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:30.736214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b418c5-a4bb-4dc2-a49a-bbbed1c33a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:30.736897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b418c5-a4bb-4dc2-a49a-bbbed1c33a6c None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:30.737176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b418c5-a4bb-4dc2-a49a-bbbed1c33a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:30.737450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b418c5-a4bb-4dc2-a49a-bbbed1c33a6c None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/345c823f8d85402ba3b07f39ed5e2fca/roles/2808f4215cc243e7a9059f529313f530` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:30.739835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b418c5-a4bb-4dc2-a49a-bbbed1c33a6c None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=345c823f8d85402ba3b07f39ed5e2fca, role_id=2808f4215cc243e7a9059f529313f530)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:30.741350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b418c5-a4bb-4dc2-a49a-bbbed1c33a6c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:30.756006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 531/1072] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:45:30 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/345c823f8d85402ba3b07f39ed5e2fca/roles/2808f4215cc243e7a9059f529313f530 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:45:31.638646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b370df-af34-4b89-91cc-e5a10dc5343f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:31.639213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b370df-af34-4b89-91cc-e5a10dc5343f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:31.639213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b370df-af34-4b89-91cc-e5a10dc5343f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:31.640747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 542/1073] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:31.646822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa39e7-5749-42bd-8d4d-697f0e23189e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:31.647309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa39e7-5749-42bd-8d4d-697f0e23189e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:31.647418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa39e7-5749-42bd-8d4d-697f0e23189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:31.665897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-67aa39e7-5749-42bd-8d4d-697f0e23189e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:31.667102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67aa39e7-5749-42bd-8d4d-697f0e23189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:31.682305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:31.741420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 532/1074] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:32.014453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b370df-af34-4b89-91cc-e5a10dc5343f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:32.044438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495650cd-6883-41ba-8e93-b863cb56393c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.044735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495650cd-6883-41ba-8e93-b863cb56393c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.045008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495650cd-6883-41ba-8e93-b863cb56393c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.046417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 533/1075] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:32.052012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb641e-30ed-4767-85fb-80f20d35ad39 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.052288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb641e-30ed-4767-85fb-80f20d35ad39 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.052560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb641e-30ed-4767-85fb-80f20d35ad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.074199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-71fb641e-30ed-4767-85fb-80f20d35ad39 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:32.075464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-71fb641e-30ed-4767-85fb-80f20d35ad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:32.096365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:32.129998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b370df-af34-4b89-91cc-e5a10dc5343f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:32.157208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 534/1076] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:32.166971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:32.167597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.167871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.168138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.178413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:32.183641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:32.184618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:32.185582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:32.189794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 543/1077] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:45:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:32.193805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb641e-30ed-4767-85fb-80f20d35ad39 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:32.275794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:32.305441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb641e-30ed-4767-85fb-80f20d35ad39 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:32.340392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:32.340996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.341273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.341541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.342228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:32.342528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:32.343871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:32.352713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 535/1078] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:45:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:32.358665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:32.435491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:32.467793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e928a-3873-4a2b-8410-b5b6a2605883 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:32.499778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:32.507769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:32.508884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.509121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.509563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] PATH_INFO: `/v3/users/791e990074424a5098dfe26384484f67` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.511070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] RBAC: Authorizing `identity:get_user(user_id=791e990074424a5098dfe26384484f67)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:32.512521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:32.514303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 544/1079] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:32 2022] GET /identity/v3/users/791e990074424a5098dfe26384484f67 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:32.520110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:32.556410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba31a3f-9c49-47e1-b04d-2c2c361a426e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:32.594911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:32.595581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.596022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.596415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.614539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:32.629547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0ca7-983a-4c11-85e9-9395e7c0e512 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:32.669693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:32.670633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.670633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.670932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.672024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] RBAC: Authorizing `identity:get_project(project_id=aae1b727e48f4bd09aa47f65e2265b30)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:32.673434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:32.675074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 545/1080] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:45:32 2022] GET /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:32.681903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:32.684714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:32.687593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:32.755883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:32.794484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e28212-3127-4fe8-a827-f45fe6e3665c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:32.808931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 536/1081] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:45:32.810493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:32.833670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d677fe-1a35-49e3-83cc-b60c5e2390be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:32.834384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d677fe-1a35-49e3-83cc-b60c5e2390be None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.834664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d677fe-1a35-49e3-83cc-b60c5e2390be None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.834937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d677fe-1a35-49e3-83cc-b60c5e2390be None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30/users/791e990074424a5098dfe26384484f67/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.837669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d677fe-1a35-49e3-83cc-b60c5e2390be None admin] RBAC: Authorizing `identity:create_grant(project_id=aae1b727e48f4bd09aa47f65e2265b30, user_id=791e990074424a5098dfe26384484f67, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:32.839142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d677fe-1a35-49e3-83cc-b60c5e2390be None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:32.855470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 546/1082] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:45:32 2022] PUT /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30/users/791e990074424a5098dfe26384484f67/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:45:32.863731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:32.931957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4447d3-dc3e-4d05-b311-0f4ca99a6662 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:32.934693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc6d57-38b6-4f5b-96e4-bfd5538c95c1 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.934986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc6d57-38b6-4f5b-96e4-bfd5538c95c1 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.935263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc6d57-38b6-4f5b-96e4-bfd5538c95c1 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.936621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 547/1083] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:32.942384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050005d-78d9-43da-bdb4-05f42edcbe3b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.942717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050005d-78d9-43da-bdb4-05f42edcbe3b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.942968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050005d-78d9-43da-bdb4-05f42edcbe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.963807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-9050005d-78d9-43da-bdb4-05f42edcbe3b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:32.965042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9050005d-78d9-43da-bdb4-05f42edcbe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:32.976669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:32.977365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:32.977653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:32.977960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:32.989585 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:32.991098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:33.011708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:33.012843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:33.013800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:33.013800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:33.013800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:33.014000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 537/1084] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:33.019642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:33.050864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 548/1085] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:33.124058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:33.137427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d26b78-c580-4634-ad1f-31da5f66cb45 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:33.173708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:33.174339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:33.174621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:33.174894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:33.175532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:33.176070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:33.177189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:33.190231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 538/1086] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:33.323838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050005d-78d9-43da-bdb4-05f42edcbe3b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:33.387333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:33.436502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050005d-78d9-43da-bdb4-05f42edcbe3b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:33.474161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:33.474800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:33.475081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:33.475925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:33.487954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:33.492862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:33.493887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:33.494788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 549/1087] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Sep 19 11:45:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:33.501728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:33.604646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388696-c6d7-494c-bb11-b3b0750d3f5d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:33.638716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:33.639241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:33.639521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:33.639792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:33.640458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:33.640760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:33.642143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:33.650777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 539/1088] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Sep 19 11:45:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:33.656434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:33.757001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09ea16-4107-46a5-a007-454e71a91d70 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:33.775498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:33.795448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:33.796067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:33.796341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:33.796608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:33.806376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:33.807789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:33.822568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:33.823659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:33.824592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:33.824592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:33.824592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:33.824592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 550/1089] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:33.829658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:33.848201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:33.926995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb645f4-5064-42f7-9577-375f1e06bb15 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:33.960007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fe853-7606-416d-9427-4047b783d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:33.960603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fe853-7606-416d-9427-4047b783d5df None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:33.960879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fe853-7606-416d-9427-4047b783d5df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:33.961173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fe853-7606-416d-9427-4047b783d5df None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:33.961829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fe853-7606-416d-9427-4047b783d5df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:33.962156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fe853-7606-416d-9427-4047b783d5df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:33.963499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fe853-7606-416d-9427-4047b783d5df None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:33.975691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 540/1090] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:33.981470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:34.079247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e8c66-0340-4ee9-a48c-40721c4f82aa None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:34.112279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:34.112903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:34.113177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:34.113443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:34.121185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:34.122626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:45:34.134099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:34.135184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:45:34.136204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 551/1091] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Sep 19 11:45:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:34.140944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fe853-7606-416d-9427-4047b783d5df None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:34.218327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:34.240599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fe853-7606-416d-9427-4047b783d5df None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:34.273316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:34.274586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:34.274861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:34.275126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:34.275842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:34.276144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:34.277549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:34.294373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 541/1092] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Sep 19 11:45:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:34.300092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:34.328257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:34.401094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd5bb0-fb9b-4fb0-b1e0-d1b79bb2fd3c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:34.436061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:34.436674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:34.436949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:34.437224 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:34.506222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:34.507572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:34.534370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:34.608056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 552/1093] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:34.613847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:34.714178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328d63-d017-40b5-9ae1-3dd1097fff2b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:34.749022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:34.749659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:34.749966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:34.750242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:34.752172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:34.762146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:34.763386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:45:34.778073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:34.779239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:45:34.780173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:45:34.780173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:45:34.780173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:34.780173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 542/1094] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:34.784873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:34.886042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d185d1-c78d-476c-ac0e-1da0ef2c8774 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:34.911845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:34.921107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7a3e5-ff99-49b3-8d59-958f98a7fba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:34.921736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7a3e5-ff99-49b3-8d59-958f98a7fba2 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:34.922048 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7a3e5-ff99-49b3-8d59-958f98a7fba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:34.922323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7a3e5-ff99-49b3-8d59-958f98a7fba2 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:34.922994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7a3e5-ff99-49b3-8d59-958f98a7fba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:34.923301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7a3e5-ff99-49b3-8d59-958f98a7fba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:34.924720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7a3e5-ff99-49b3-8d59-958f98a7fba2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:34.939571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 553/1095] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:34.945610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:35.048105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bf548-40b9-428c-a422-a93d17c2b45d None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.048388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bf548-40b9-428c-a422-a93d17c2b45d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.048671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bf548-40b9-428c-a422-a93d17c2b45d None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.050057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 554/1096] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:35.053196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b767afd-fbe1-4d33-b707-454a003c13fe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:35.054896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c2fdf-2b70-4cbf-b933-93a337f286ce None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.055723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c2fdf-2b70-4cbf-b933-93a337f286ce None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.056004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c2fdf-2b70-4cbf-b933-93a337f286ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.073963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fa6c2fdf-2b70-4cbf-b933-93a337f286ce None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:35.075235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fa6c2fdf-2b70-4cbf-b933-93a337f286ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:35.088703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:35.089339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.089614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.089923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.096679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:35.098142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:45:35.109643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:35.110715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:35.111662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 543/1097] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:35.116886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:35.149901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 555/1098] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:35.164868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:35.223510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011f272-d6c7-440e-bef7-38d4f37ed5e6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:35.258215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:35.258818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.259099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.259373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.260060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:35.260369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:35.261802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:35.274919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 544/1099] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:35.281405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c2fdf-2b70-4cbf-b933-93a337f286ce None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:35.395519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c2fdf-2b70-4cbf-b933-93a337f286ce None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:35.412176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:35.413998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:35.433031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:35.434699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.434699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.434699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.435019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] RBAC: Adding query filter params (role.id=2808f4215cc243e7a9059f529313f530, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=345c823f8d85402ba3b07f39ed5e2fca) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:35.435206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:35.436627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:35.442877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 556/1100] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:35 2022] GET /identity/v3/role_assignments?user.id=345c823f8d85402ba3b07f39ed5e2fca&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=2808f4215cc243e7a9059f529313f530 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:35.565181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f77f8-ac0d-4181-b0c8-556975b7a381 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:35.574593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:35.607224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:35.607884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.608171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.608444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.619795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:35.624678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:35.625694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:35.626712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 545/1101] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:45:35.632503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:35.743222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dde7f-8dab-4114-91ad-f4cb15cb5168 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:35.779905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:35.780519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.780799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.781078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.781751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:35.782076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:35.783426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:35.793246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 557/1102] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:45:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:35.799468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:35.909507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e0b45-01f8-4afd-97d9-8e0343b99f3d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:35.946160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:35.946785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:35.947057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:35.947331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] PATH_INFO: `/v3/users/791e990074424a5098dfe26384484f67` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:35.948453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] RBAC: Authorizing `identity:get_user(user_id=791e990074424a5098dfe26384484f67)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:35.949922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:35.951508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 546/1103] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:35 2022] GET /identity/v3/users/791e990074424a5098dfe26384484f67 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:35.957688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:36.006113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:36.069625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5349386-2421-4a5a-9afd-7bd554c114a8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:36.104581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:36.105188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:36.105464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:36.105733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:36.181912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:36.183344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:36.232013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:36.292594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 558/1104] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:36.299307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:36.335687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:36.415226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aeccb-718b-483d-aa7f-6ff3e599db73 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:36.448163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c129ce-0294-4c58-915a-154e8edc76d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:36.448780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c129ce-0294-4c58-915a-154e8edc76d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:36.449059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c129ce-0294-4c58-915a-154e8edc76d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:36.449333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c129ce-0294-4c58-915a-154e8edc76d3 None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:36.450461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c129ce-0294-4c58-915a-154e8edc76d3 None admin] RBAC: Authorizing `identity:get_project(project_id=aae1b727e48f4bd09aa47f65e2265b30)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:36.451888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c129ce-0294-4c58-915a-154e8edc76d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:36.453745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 547/1105] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:45:36 2022] GET /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:36.459915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:36.556641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa67c2b-c5a2-4d84-89d0-2f8571b39f3b None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:36.591901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d85e7-3ff9-4f26-aa83-e27e92c5b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:36.592493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d85e7-3ff9-4f26-aa83-e27e92c5b020 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:36.592770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d85e7-3ff9-4f26-aa83-e27e92c5b020 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:36.593039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d85e7-3ff9-4f26-aa83-e27e92c5b020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:36.593782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d85e7-3ff9-4f26-aa83-e27e92c5b020 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=aae1b727e48f4bd09aa47f65e2265b30, user.id=791e990074424a5098dfe26384484f67) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:36.594092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d85e7-3ff9-4f26-aa83-e27e92c5b020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:36.595493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d85e7-3ff9-4f26-aa83-e27e92c5b020 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:36.601009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 559/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:36 2022] GET /identity/v3/role_assignments?user.id=791e990074424a5098dfe26384484f67&scope.project.id=aae1b727e48f4bd09aa47f65e2265b30&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:38.779707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67b142-5fb8-4357-b776-f863bec6552c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:38.780270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67b142-5fb8-4357-b776-f863bec6552c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:38.780270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67b142-5fb8-4357-b776-f863bec6552c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:38.781641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 548/1107] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:38.786836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9483b-746b-4fce-a61c-b1d10a321cc1 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:38.787214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9483b-746b-4fce-a61c-b1d10a321cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:38.787385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9483b-746b-4fce-a61c-b1d10a321cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:38.805098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a9e9483b-746b-4fce-a61c-b1d10a321cc1 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:38.806319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a9e9483b-746b-4fce-a61c-b1d10a321cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:38.808251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:38.871514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 560/1108] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:39.141509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67b142-5fb8-4357-b776-f863bec6552c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:39.240203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67b142-5fb8-4357-b776-f863bec6552c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:39.241680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:39.275220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbb017-91b5-4170-94d9-f0b6a02e9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:39.275841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb017-91b5-4170-94d9-f0b6a02e9c86 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:39.276121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb017-91b5-4170-94d9-f0b6a02e9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:39.276394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb017-91b5-4170-94d9-f0b6a02e9c86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:39.277494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbb017-91b5-4170-94d9-f0b6a02e9c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:39.278961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbb017-91b5-4170-94d9-f0b6a02e9c86 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:39.280517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 549/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:39.286288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9483b-746b-4fce-a61c-b1d10a321cc1 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:39.382493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9483b-746b-4fce-a61c-b1d10a321cc1 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:39.415377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b93fa9-48b3-4a9d-b2bb-c9f864129bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:39.415908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b93fa9-48b3-4a9d-b2bb-c9f864129bed None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:39.416185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b93fa9-48b3-4a9d-b2bb-c9f864129bed None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:39.416456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b93fa9-48b3-4a9d-b2bb-c9f864129bed None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:39.417099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b93fa9-48b3-4a9d-b2bb-c9f864129bed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:39.418528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b93fa9-48b3-4a9d-b2bb-c9f864129bed None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:39.428376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:39.441814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 561/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:45:39 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:41.614321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249064-d65b-44de-ab54-5f3779ac8210 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:41.614853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249064-d65b-44de-ab54-5f3779ac8210 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:41.614853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249064-d65b-44de-ab54-5f3779ac8210 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:41.616207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 550/1111] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:41.622257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ac315-9557-4847-848c-2994fc91285d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:41.622669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ac315-9557-4847-848c-2994fc91285d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:41.622800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ac315-9557-4847-848c-2994fc91285d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:41.641163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8e2ac315-9557-4847-848c-2994fc91285d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:41.642366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8e2ac315-9557-4847-848c-2994fc91285d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:41.706094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 562/1112] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:41.982351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249064-d65b-44de-ab54-5f3779ac8210 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:42.080512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249064-d65b-44de-ab54-5f3779ac8210 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:42.114042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:42.114651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:42.114953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:42.115222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] PATH_INFO: `/v3/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:42.116571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] RBAC: Authorizing `identity:get_role(role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:42.117948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:42.119245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 551/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:41 2022] GET /identity/v3/roles/0a108a4463a4458480b00aed843b4e9b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:42.125021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ac315-9557-4847-848c-2994fc91285d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:42.219716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ac315-9557-4847-848c-2994fc91285d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:42.242159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:42.253143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:42.253781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:42.254068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:42.254334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] PATH_INFO: `/v3/users/c44fb14678a34679999b3537e066bf71` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:42.266893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] RBAC: Authorizing `identity:get_user(user_id=c44fb14678a34679999b3537e066bf71)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:42.268304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:42.269900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 563/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:42 2022] GET /identity/v3/users/c44fb14678a34679999b3537e066bf71 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:42.276280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:42.316725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:42.375916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7f000-c8c5-48df-92bf-10c1ab481842 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:42.409348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:42.409956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:42.410230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:42.410497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:42.474307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:42.475720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:42.513294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:42.572401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 552/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:42.578756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:42.675305 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048793a3-2d51-4700-ae97-4cf4de66938e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:42.709283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25917ab5-ce2d-48ab-a79c-656a382e8fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:42.709935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25917ab5-ce2d-48ab-a79c-656a382e8fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:42.710218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25917ab5-ce2d-48ab-a79c-656a382e8fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:42.710495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25917ab5-ce2d-48ab-a79c-656a382e8fc3 None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:42.711639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25917ab5-ce2d-48ab-a79c-656a382e8fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=aae1b727e48f4bd09aa47f65e2265b30)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:42.713051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25917ab5-ce2d-48ab-a79c-656a382e8fc3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:42.714703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 564/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:45:42 2022] GET /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:42.721074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:42.826342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f12c8d-5e0a-4cc3-b0fc-574ebd5cc7dc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:42.858570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c7260-bc79-462f-ab47-c91b5bc29f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:42.859168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c7260-bc79-462f-ab47-c91b5bc29f1f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:42.859448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c7260-bc79-462f-ab47-c91b5bc29f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:42.859719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c7260-bc79-462f-ab47-c91b5bc29f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:42.860440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c7260-bc79-462f-ab47-c91b5bc29f1f None admin] RBAC: Adding query filter params (role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=aae1b727e48f4bd09aa47f65e2265b30, user.id=c44fb14678a34679999b3537e066bf71) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:42.860751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c7260-bc79-462f-ab47-c91b5bc29f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:42.862234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c7260-bc79-462f-ab47-c91b5bc29f1f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:42.867915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 553/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:42 2022] GET /identity/v3/role_assignments?user.id=c44fb14678a34679999b3537e066bf71&scope.project.id=aae1b727e48f4bd09aa47f65e2265b30&role.id=0a108a4463a4458480b00aed843b4e9b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:44.979029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090cd2d-a169-42b6-8890-a77084524e2c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:44.979592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090cd2d-a169-42b6-8890-a77084524e2c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:44.979592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090cd2d-a169-42b6-8890-a77084524e2c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:44.980961 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 565/1118] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:44.986214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360c6a-8e0e-47d4-9e0b-1ff55fdfb438 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:44.986598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360c6a-8e0e-47d4-9e0b-1ff55fdfb438 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:44.986783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360c6a-8e0e-47d4-9e0b-1ff55fdfb438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:44.993447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:45.005938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5c360c6a-8e0e-47d4-9e0b-1ff55fdfb438 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:45.007126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c360c6a-8e0e-47d4-9e0b-1ff55fdfb438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:45.076741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 554/1119] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:45.350083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090cd2d-a169-42b6-8890-a77084524e2c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:45.401256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:45.449334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090cd2d-a169-42b6-8890-a77084524e2c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:45.482036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:45.482656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:45.482930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:45.483197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] PATH_INFO: `/v3/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:45.484574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] RBAC: Authorizing `identity:get_role(role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:45.485936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:45.487239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 566/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:45 2022] GET /identity/v3/roles/0a108a4463a4458480b00aed843b4e9b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:45.493117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c360c6a-8e0e-47d4-9e0b-1ff55fdfb438 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:45.611610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c360c6a-8e0e-47d4-9e0b-1ff55fdfb438 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:45.640681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:45.646080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:45.646704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:45.646984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:45.647261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] PATH_INFO: `/v3/users/c44fb14678a34679999b3537e066bf71` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:45.649037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=c44fb14678a34679999b3537e066bf71)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:45.650537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:45.652250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 555/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:45 2022] GET /identity/v3/users/c44fb14678a34679999b3537e066bf71 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:45.658096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:45.754713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aab368-827c-4bb2-897b-ba61305c18fe None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:45.771158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:45.787434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733647d1-d302-4591-a5f4-9911f80d019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:45.788048 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733647d1-d302-4591-a5f4-9911f80d019c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:45.788325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733647d1-d302-4591-a5f4-9911f80d019c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:45.788594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733647d1-d302-4591-a5f4-9911f80d019c None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:45.789711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733647d1-d302-4591-a5f4-9911f80d019c None admin] RBAC: Authorizing `identity:get_project(project_id=aae1b727e48f4bd09aa47f65e2265b30)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:45.791135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733647d1-d302-4591-a5f4-9911f80d019c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:45.792730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 567/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:45:45 2022] GET /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:45:45.798875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:45.926029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4b71-d108-4660-bbdd-3f9c938a7ea0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:45.960341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effc262-b357-42b5-a023-f0df77b9c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:45.961009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effc262-b357-42b5-a023-f0df77b9c79b None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:45.961286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effc262-b357-42b5-a023-f0df77b9c79b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:45.961555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effc262-b357-42b5-a023-f0df77b9c79b None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30/users/c44fb14678a34679999b3537e066bf71/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:45.963658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effc262-b357-42b5-a023-f0df77b9c79b None admin] RBAC: Authorizing `identity:create_grant(project_id=aae1b727e48f4bd09aa47f65e2265b30, user_id=c44fb14678a34679999b3537e066bf71, role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:45.965069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effc262-b357-42b5-a023-f0df77b9c79b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:45.978560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 556/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:45:45 2022] PUT /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30/users/c44fb14678a34679999b3537e066bf71/roles/0a108a4463a4458480b00aed843b4e9b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:45:48.104298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e5325-9019-46aa-9067-efadd7682ed6 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:48.104882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e5325-9019-46aa-9067-efadd7682ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:48.104882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e5325-9019-46aa-9067-efadd7682ed6 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:48.106195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 568/1124] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:45:48.111336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20d455-c72b-456a-ab2d-eee7999a2f1e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:48.111741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20d455-c72b-456a-ab2d-eee7999a2f1e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:48.111880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20d455-c72b-456a-ab2d-eee7999a2f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:48.130472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bc20d455-c72b-456a-ab2d-eee7999a2f1e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:48.131657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bc20d455-c72b-456a-ab2d-eee7999a2f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:48.188188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:48.201509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 557/1125] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:48.468076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e5325-9019-46aa-9067-efadd7682ed6 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:48.567314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e5325-9019-46aa-9067-efadd7682ed6 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:48.599591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:48.600203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:48.600477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:48.600752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] PATH_INFO: `/v3/roles/0a108a4463a4458480b00aed843b4e9b` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:48.602708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] RBAC: Authorizing `identity:get_role(role_id=0a108a4463a4458480b00aed843b4e9b)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:48.604126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:48.605429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 569/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:48 2022] GET /identity/v3/roles/0a108a4463a4458480b00aed843b4e9b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:48.611293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20d455-c72b-456a-ab2d-eee7999a2f1e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:48.711880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20d455-c72b-456a-ab2d-eee7999a2f1e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:48.747272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:48.747851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:48.748153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:48.748462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] PATH_INFO: `/v3/users/c44fb14678a34679999b3537e066bf71` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:48.749598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] RBAC: Authorizing `identity:get_user(user_id=c44fb14678a34679999b3537e066bf71)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:48.751065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:48.752785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 558/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:48 2022] GET /identity/v3/users/c44fb14678a34679999b3537e066bf71 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:48.758591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:48.813160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:48.856865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c72e1-5276-457b-a797-39fa678fd1b3 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:48.889199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:48.889817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:48.890098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:48.890376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:48.957619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:48.959075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:49.011343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:49.055083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 570/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:49.063477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:49.124667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:49.165646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e5b4c-5e74-47f3-a8c0-0a6045056b0e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:49.199677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f283e-1870-4001-96d4-fab841ea0d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:49.200302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f283e-1870-4001-96d4-fab841ea0d83 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:49.200591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f283e-1870-4001-96d4-fab841ea0d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:49.200865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f283e-1870-4001-96d4-fab841ea0d83 None admin] PATH_INFO: `/v3/projects/aae1b727e48f4bd09aa47f65e2265b30` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:49.202064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f283e-1870-4001-96d4-fab841ea0d83 None admin] RBAC: Authorizing `identity:get_project(project_id=aae1b727e48f4bd09aa47f65e2265b30)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:49.203710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f283e-1870-4001-96d4-fab841ea0d83 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:49.205333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 559/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:45:49 2022] GET /identity/v3/projects/aae1b727e48f4bd09aa47f65e2265b30 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:49.211497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:49.308438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454f0b-36c0-404a-8709-b5755aca912d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:49.342312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cf056-87fb-4e0a-93d8-f8bd48ece8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:49.342902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cf056-87fb-4e0a-93d8-f8bd48ece8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:49.343182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cf056-87fb-4e0a-93d8-f8bd48ece8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:49.343453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cf056-87fb-4e0a-93d8-f8bd48ece8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:49.344178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cf056-87fb-4e0a-93d8-f8bd48ece8f4 None admin] RBAC: Adding query filter params (role.id=0a108a4463a4458480b00aed843b4e9b, scope.project.id=aae1b727e48f4bd09aa47f65e2265b30, user.id=c44fb14678a34679999b3537e066bf71) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:49.344484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cf056-87fb-4e0a-93d8-f8bd48ece8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:49.345917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cf056-87fb-4e0a-93d8-f8bd48ece8f4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:49.351568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 571/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:49 2022] GET /identity/v3/role_assignments?user.id=c44fb14678a34679999b3537e066bf71&scope.project.id=aae1b727e48f4bd09aa47f65e2265b30&role.id=0a108a4463a4458480b00aed843b4e9b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:51.573234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430071-81b0-4b4e-b50b-ae8a3012ecaa None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:51.573904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430071-81b0-4b4e-b50b-ae8a3012ecaa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:51.573904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430071-81b0-4b4e-b50b-ae8a3012ecaa None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:51.575137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 560/1131] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:51.582027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852017ab-52e4-491a-be88-6f0f589b0ccd None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:51.582881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852017ab-52e4-491a-be88-6f0f589b0ccd None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:51.582881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852017ab-52e4-491a-be88-6f0f589b0ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:51.606040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-852017ab-52e4-491a-be88-6f0f589b0ccd None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:51.607238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-852017ab-52e4-491a-be88-6f0f589b0ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:51.621777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:51.681357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 572/1132] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:45:51.950821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73430071-81b0-4b4e-b50b-ae8a3012ecaa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:52.048099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73430071-81b0-4b4e-b50b-ae8a3012ecaa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:52.074727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:52.083789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27852f6-4833-4aaf-a357-a6152ac743b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:52.084415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27852f6-4833-4aaf-a357-a6152ac743b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:52.084692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27852f6-4833-4aaf-a357-a6152ac743b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:52.084960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27852f6-4833-4aaf-a357-a6152ac743b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:52.086104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27852f6-4833-4aaf-a357-a6152ac743b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:52.087524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27852f6-4833-4aaf-a357-a6152ac743b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:52.089070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 561/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:52.094969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852017ab-52e4-491a-be88-6f0f589b0ccd None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:52.194682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852017ab-52e4-491a-be88-6f0f589b0ccd None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:52.231887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55a6c1-bd80-4692-a9bc-24bf275346c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:52.232465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55a6c1-bd80-4692-a9bc-24bf275346c2 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:52.232755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55a6c1-bd80-4692-a9bc-24bf275346c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:52.233063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55a6c1-bd80-4692-a9bc-24bf275346c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:52.233807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55a6c1-bd80-4692-a9bc-24bf275346c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:52.235334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55a6c1-bd80-4692-a9bc-24bf275346c2 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:52.256830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:52.267561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 573/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:45:52 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:54.582733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4de8-ab0b-442e-b358-53c1441c3376 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:54.583547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4de8-ab0b-442e-b358-53c1441c3376 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:54.583547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4de8-ab0b-442e-b358-53c1441c3376 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:54.584632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 562/1135] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:54.591243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59673-5556-40c8-a70d-a7f749f4ba23 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:54.591731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59673-5556-40c8-a70d-a7f749f4ba23 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:54.591824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59673-5556-40c8-a70d-a7f749f4ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:54.610520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b3c59673-5556-40c8-a70d-a7f749f4ba23 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:54.611716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b3c59673-5556-40c8-a70d-a7f749f4ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:54.678288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 574/1136] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:54.950811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b4de8-ab0b-442e-b358-53c1441c3376 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:55.056351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b4de8-ab0b-442e-b358-53c1441c3376 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:55.091049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee369f-a116-4e85-87fb-1eecac35c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:55.091609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee369f-a116-4e85-87fb-1eecac35c33c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:55.091927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee369f-a116-4e85-87fb-1eecac35c33c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:55.092202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee369f-a116-4e85-87fb-1eecac35c33c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:55.093388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee369f-a116-4e85-87fb-1eecac35c33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:55.094970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee369f-a116-4e85-87fb-1eecac35c33c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:55.096594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 563/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:45:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:55.103118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c59673-5556-40c8-a70d-a7f749f4ba23 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:55.200998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c59673-5556-40c8-a70d-a7f749f4ba23 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:55.231091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:55.237316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78af040-2ce8-4041-85bc-dc151f2e2bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:55.237967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78af040-2ce8-4041-85bc-dc151f2e2bc1 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:55.238249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78af040-2ce8-4041-85bc-dc151f2e2bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:55.238523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78af040-2ce8-4041-85bc-dc151f2e2bc1 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:55.239190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78af040-2ce8-4041-85bc-dc151f2e2bc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:55.240604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78af040-2ce8-4041-85bc-dc151f2e2bc1 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:55.263017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 575/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:45:55 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:45:57.443777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4922-d12b-4284-a3d4-de7512b9848f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:57.444331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4922-d12b-4284-a3d4-de7512b9848f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:57.444331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4922-d12b-4284-a3d4-de7512b9848f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:57.446093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 564/1139] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:45:57.451535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef14f8f-4c28-4fd7-a666-24f13778b03f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:57.451948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef14f8f-4c28-4fd7-a666-24f13778b03f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:57.452088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef14f8f-4c28-4fd7-a666-24f13778b03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:57.470766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fef14f8f-4c28-4fd7-a666-24f13778b03f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:45:57.471946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fef14f8f-4c28-4fd7-a666-24f13778b03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:57.536281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 576/1140] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:45:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:45:57.807920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4922-d12b-4284-a3d4-de7512b9848f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:57.870424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:57.909988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4922-d12b-4284-a3d4-de7512b9848f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:57.944034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:57.944680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:57.944956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:57.945226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:57.955194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:45:57.959905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:45:57.961574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:45:57.962825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 565/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:45:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:45:57.967841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef14f8f-4c28-4fd7-a666-24f13778b03f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:58.062662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef14f8f-4c28-4fd7-a666-24f13778b03f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:58.078757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:58.097236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:58.097863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:58.098137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:58.098406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:58.099056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:58.099353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:58.100693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:58.108416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 577/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:45:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:58.114526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:58.210937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340c1e8-6cf3-4817-a60b-0a745a5e7920 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:58.233054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:58.245418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:58.246070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:58.246349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:58.246621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] PATH_INFO: `/v3/users/c7c18ef39f0c40b4ae61875da9547ee7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:58.259330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] RBAC: Authorizing `identity:get_user(user_id=c7c18ef39f0c40b4ae61875da9547ee7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:58.260752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:58.262373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 566/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:45:58 2022] GET /identity/v3/users/c7c18ef39f0c40b4ae61875da9547ee7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:58.268405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:58.364908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dcc58-6ddf-453c-98e8-0901f8402c6c None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:58.397735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:58.398358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:58.398638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:58.398910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:58.443275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:58.465821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:58.467245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:58.562830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 578/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:45:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:45:58.569170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:58.668380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51f812-a97b-4421-882e-24265b3886a5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:58.700909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805ff40-d234-456d-9555-c42938189a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:58.701527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805ff40-d234-456d-9555-c42938189a37 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:58.701831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805ff40-d234-456d-9555-c42938189a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:58.702109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805ff40-d234-456d-9555-c42938189a37 None admin] PATH_INFO: `/v3/projects/7c91660791784d51bd7c20b6fde16d94` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:58.703202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805ff40-d234-456d-9555-c42938189a37 None admin] RBAC: Authorizing `identity:get_project(project_id=7c91660791784d51bd7c20b6fde16d94)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:58.704887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805ff40-d234-456d-9555-c42938189a37 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:58.706528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 567/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:45:58 2022] GET /identity/v3/projects/7c91660791784d51bd7c20b6fde16d94 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:45:58.713164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:58.795418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:58.817153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bdfd0-9c4e-4761-bdbf-54c66b1c2179 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:58.852116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dd33b-462d-4cac-84b4-f8b7e1c6ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:58.852710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd33b-462d-4cac-84b4-f8b7e1c6ff17 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:58.852991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd33b-462d-4cac-84b4-f8b7e1c6ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:58.853267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd33b-462d-4cac-84b4-f8b7e1c6ff17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:58.854022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dd33b-462d-4cac-84b4-f8b7e1c6ff17 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=7c91660791784d51bd7c20b6fde16d94, user.id=c7c18ef39f0c40b4ae61875da9547ee7) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:45:58.854332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dd33b-462d-4cac-84b4-f8b7e1c6ff17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:58.855745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dd33b-462d-4cac-84b4-f8b7e1c6ff17 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:58.861308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 579/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:45:58 2022] GET /identity/v3/role_assignments?user.id=c7c18ef39f0c40b4ae61875da9547ee7&scope.project.id=7c91660791784d51bd7c20b6fde16d94&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:01.063221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acecf-fcfe-48d6-b9bb-6fa5869306ca None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:01.063782 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acecf-fcfe-48d6-b9bb-6fa5869306ca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:01.063782 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acecf-fcfe-48d6-b9bb-6fa5869306ca None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:01.065083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 568/1147] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:01.070546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358d5e-4ad0-49c0-8e41-18d21ed236c3 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:01.070937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358d5e-4ad0-49c0-8e41-18d21ed236c3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:01.071135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358d5e-4ad0-49c0-8e41-18d21ed236c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:01.090019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2f358d5e-4ad0-49c0-8e41-18d21ed236c3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:01.091160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2f358d5e-4ad0-49c0-8e41-18d21ed236c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:01.175735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 580/1148] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:46:01.449495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27acecf-fcfe-48d6-b9bb-6fa5869306ca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:01.504839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:01.560991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27acecf-fcfe-48d6-b9bb-6fa5869306ca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:01.598090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:01.598752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:01.598992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:01.599435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:01.612195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:46:01.617136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:46:01.618389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:46:01.619728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 569/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:46:01.624444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f358d5e-4ad0-49c0-8e41-18d21ed236c3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:01.698770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:01.730697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f358d5e-4ad0-49c0-8e41-18d21ed236c3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:01.764722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:01.765329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:01.765602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:01.765898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:01.766562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:01.766860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:01.768206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:01.777834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 581/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:01.783981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:01.882748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324dd99-2952-413d-96df-c938408271a8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:01.891966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:01.918850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:01.919488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:01.919766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:01.920042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] PATH_INFO: `/v3/users/c7c18ef39f0c40b4ae61875da9547ee7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:01.921154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] RBAC: Authorizing `identity:get_user(user_id=c7c18ef39f0c40b4ae61875da9547ee7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:01.922613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:01.924187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 570/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:46:01 2022] GET /identity/v3/users/c7c18ef39f0c40b4ae61875da9547ee7 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:01.930668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:02.027849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82957e-cc96-4ce6-9172-aa29fcaef774 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:02.064451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d1286-958c-43cc-8c0a-df0b643e86f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:02.065081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d1286-958c-43cc-8c0a-df0b643e86f3 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:02.065363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d1286-958c-43cc-8c0a-df0b643e86f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:02.065641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d1286-958c-43cc-8c0a-df0b643e86f3 None admin] PATH_INFO: `/v3/projects/7c91660791784d51bd7c20b6fde16d94` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:02.066796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d1286-958c-43cc-8c0a-df0b643e86f3 None admin] RBAC: Authorizing `identity:get_project(project_id=7c91660791784d51bd7c20b6fde16d94)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:02.068217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d1286-958c-43cc-8c0a-df0b643e86f3 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:02.070040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 582/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:46:01 2022] GET /identity/v3/projects/7c91660791784d51bd7c20b6fde16d94 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:02.075959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:02.178505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a747b1c-e14c-49e6-b449-9db12c4cad51 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:02.214390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301d46e-4645-4f2f-8c95-bc4424467561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:02.215042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301d46e-4645-4f2f-8c95-bc4424467561 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:02.215322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301d46e-4645-4f2f-8c95-bc4424467561 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:02.215596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301d46e-4645-4f2f-8c95-bc4424467561 None admin] PATH_INFO: `/v3/projects/7c91660791784d51bd7c20b6fde16d94/users/c7c18ef39f0c40b4ae61875da9547ee7/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:02.217568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301d46e-4645-4f2f-8c95-bc4424467561 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c91660791784d51bd7c20b6fde16d94, user_id=c7c18ef39f0c40b4ae61875da9547ee7, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:02.219055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301d46e-4645-4f2f-8c95-bc4424467561 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:02.233013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 571/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:46:02 2022] PUT /identity/v3/projects/7c91660791784d51bd7c20b6fde16d94/users/c7c18ef39f0c40b4ae61875da9547ee7/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:46:04.403231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708cb5-6f92-41cc-8d72-e3882a6fc07b None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:04.403898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708cb5-6f92-41cc-8d72-e3882a6fc07b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:04.403898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708cb5-6f92-41cc-8d72-e3882a6fc07b None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:04.404967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 583/1154] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:04.410680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824767f0-6157-44c4-871e-1a9faca686de None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:04.411158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824767f0-6157-44c4-871e-1a9faca686de None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:04.411273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824767f0-6157-44c4-871e-1a9faca686de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:04.431550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-824767f0-6157-44c4-871e-1a9faca686de None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:04.432835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-824767f0-6157-44c4-871e-1a9faca686de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:04.464936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:04.513580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 572/1155] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:04.798379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7708cb5-6f92-41cc-8d72-e3882a6fc07b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:04.801269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:04.906743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7708cb5-6f92-41cc-8d72-e3882a6fc07b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:04.939731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:04.940352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:04.940631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:04.940902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:04.952226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:46:04.957250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:46:04.958667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:46:04.959847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 584/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:46:04.966149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824767f0-6157-44c4-871e-1a9faca686de None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:05.063621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824767f0-6157-44c4-871e-1a9faca686de None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:05.098444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:05.099049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:05.099326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:05.099598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:05.100261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:05.100595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:05.101972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:05.109738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 573/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:05.116133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:05.177182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:05.218192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d54b93-aa77-4b6a-b18d-e352200241dc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:05.252557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:05.253170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:05.253451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:05.253723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] PATH_INFO: `/v3/users/c7c18ef39f0c40b4ae61875da9547ee7` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:05.255030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] RBAC: Authorizing `identity:get_user(user_id=c7c18ef39f0c40b4ae61875da9547ee7)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:05.256531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:05.258791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 585/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:46:05 2022] GET /identity/v3/users/c7c18ef39f0c40b4ae61875da9547ee7 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:05.265000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:05.296550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:05.371495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46c013-9ed5-4203-ab41-7eaa87e6d1c0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:05.404321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:05.404917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:05.405228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:05.405467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:05.470299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:05.471726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:05.497694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:05.567645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 574/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:46:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:46:05.573837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:05.670897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2746b-aea6-461f-87ce-9ee7a244cb12 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:05.698471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:05.704982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73350320-629a-479b-8b23-9e7ef7eff399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:05.705662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350320-629a-479b-8b23-9e7ef7eff399 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:05.705999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350320-629a-479b-8b23-9e7ef7eff399 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:05.706296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350320-629a-479b-8b23-9e7ef7eff399 None admin] PATH_INFO: `/v3/projects/7c91660791784d51bd7c20b6fde16d94` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:05.707431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73350320-629a-479b-8b23-9e7ef7eff399 None admin] RBAC: Authorizing `identity:get_project(project_id=7c91660791784d51bd7c20b6fde16d94)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:05.708846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73350320-629a-479b-8b23-9e7ef7eff399 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:05.710662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 586/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:46:05 2022] GET /identity/v3/projects/7c91660791784d51bd7c20b6fde16d94 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:05.716887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:05.814823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1b20a-d6df-4f39-b7e2-52cfa3f8a6f4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:05.842248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:05.850206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ccae3-938c-4ffd-9cc1-63c73d52a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:05.851355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ccae3-938c-4ffd-9cc1-63c73d52a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:05.851636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ccae3-938c-4ffd-9cc1-63c73d52a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:05.851911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ccae3-938c-4ffd-9cc1-63c73d52a9c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:05.852638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ccae3-938c-4ffd-9cc1-63c73d52a9c9 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=7c91660791784d51bd7c20b6fde16d94, user.id=c7c18ef39f0c40b4ae61875da9547ee7) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:05.852950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ccae3-938c-4ffd-9cc1-63c73d52a9c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:05.854462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ccae3-938c-4ffd-9cc1-63c73d52a9c9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:05.860592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 575/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:46:05 2022] GET /identity/v3/role_assignments?user.id=c7c18ef39f0c40b4ae61875da9547ee7&scope.project.id=7c91660791784d51bd7c20b6fde16d94&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:08.063776 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9e6cb-5d07-40cd-8b63-be750b94a197 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:08.064340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9e6cb-5d07-40cd-8b63-be750b94a197 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:08.064340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9e6cb-5d07-40cd-8b63-be750b94a197 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:08.065612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 587/1162] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:08.070978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ab656-f00c-495c-90d6-fea9548b31db None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:08.071380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ab656-f00c-495c-90d6-fea9548b31db None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:08.071525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ab656-f00c-495c-90d6-fea9548b31db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:08.089791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8a3ab656-f00c-495c-90d6-fea9548b31db None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:08.090989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8a3ab656-f00c-495c-90d6-fea9548b31db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:08.158925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 576/1163] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:08.437609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9e6cb-5d07-40cd-8b63-be750b94a197 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:08.543154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9e6cb-5d07-40cd-8b63-be750b94a197 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:08.578964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:08.579578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:08.579857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:08.580126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:08.647969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:08.651691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:08.653308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:08.754171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 588/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:46:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:46:08.760686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ab656-f00c-495c-90d6-fea9548b31db None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:08.880345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ab656-f00c-495c-90d6-fea9548b31db None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:08.907907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:08.921431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733b35-3872-44ba-8045-2d47beb79376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:08.922098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733b35-3872-44ba-8045-2d47beb79376 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:08.922382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733b35-3872-44ba-8045-2d47beb79376 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:08.922654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733b35-3872-44ba-8045-2d47beb79376 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:08.930512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733b35-3872-44ba-8045-2d47beb79376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:08.931962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733b35-3872-44ba-8045-2d47beb79376 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-49733b35-3872-44ba-8045-2d47beb79376 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:46:08.944955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:46:08.947175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 11:46:08.949178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:46:08.951183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 577/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:46:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:46:08.952344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:09.050653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:09.056132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd24c2-5e54-4857-b6dd-cb116739f63d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:09.100369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3bb1f-d6b9-4746-8ae1-ad88e78c2e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:09.101078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3bb1f-d6b9-4746-8ae1-ad88e78c2e1c None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:09.101230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3bb1f-d6b9-4746-8ae1-ad88e78c2e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:09.101499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3bb1f-d6b9-4746-8ae1-ad88e78c2e1c None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:09.102170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3bb1f-d6b9-4746-8ae1-ad88e78c2e1c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:09.102496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3bb1f-d6b9-4746-8ae1-ad88e78c2e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:09.103838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3bb1f-d6b9-4746-8ae1-ad88e78c2e1c None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:09.111266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 589/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Sep 19 11:46:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:11.322764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6a7ac-4843-45f7-9784-088f1d4ce4df None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:11.323322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6a7ac-4843-45f7-9784-088f1d4ce4df None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:11.323322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6a7ac-4843-45f7-9784-088f1d4ce4df None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:11.324643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 578/1167] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:11.330356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49d965-4ae3-43f4-807a-e3d90e7c66af None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:11.330841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49d965-4ae3-43f4-807a-e3d90e7c66af None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:11.330938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49d965-4ae3-43f4-807a-e3d90e7c66af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:11.349569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4f49d965-4ae3-43f4-807a-e3d90e7c66af None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:11.350976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4f49d965-4ae3-43f4-807a-e3d90e7c66af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:11.443111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 590/1168] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:11.715264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6a7ac-4843-45f7-9784-088f1d4ce4df None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:11.821110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6a7ac-4843-45f7-9784-088f1d4ce4df None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:11.858364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa24fd-40a7-4c46-abb1-733ee675291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:11.858975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa24fd-40a7-4c46-abb1-733ee675291b None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:11.859253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa24fd-40a7-4c46-abb1-733ee675291b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:11.859526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa24fd-40a7-4c46-abb1-733ee675291b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:11.860208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa24fd-40a7-4c46-abb1-733ee675291b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:11.861586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa24fd-40a7-4c46-abb1-733ee675291b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:11.883436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 579/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Mon Sep 19 11:46:11 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:46:14.177223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c9dc5-dc83-481b-9f2f-2fbbd08a657a None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:14.177798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c9dc5-dc83-481b-9f2f-2fbbd08a657a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:14.177798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c9dc5-dc83-481b-9f2f-2fbbd08a657a None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:14.179729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 591/1170] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:14.185257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be22816-b2de-4d8e-822c-13f568a69af9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:14.185719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be22816-b2de-4d8e-822c-13f568a69af9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:14.185871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be22816-b2de-4d8e-822c-13f568a69af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:14.204428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4be22816-b2de-4d8e-822c-13f568a69af9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:14.205632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4be22816-b2de-4d8e-822c-13f568a69af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:14.248638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:14.282597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 580/1171] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:14.551442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c9dc5-dc83-481b-9f2f-2fbbd08a657a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:14.650208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:14.663576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c9dc5-dc83-481b-9f2f-2fbbd08a657a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:14.708064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8d8cf-1f5b-4171-8a76-4b70061b8d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:14.708701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8d8cf-1f5b-4171-8a76-4b70061b8d99 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:14.708980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8d8cf-1f5b-4171-8a76-4b70061b8d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:14.709252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8d8cf-1f5b-4171-8a76-4b70061b8d99 None admin] PATH_INFO: `/v3/domains/04298a5ae3de4c6ea19be2c49048b771` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:14.721082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8d8cf-1f5b-4171-8a76-4b70061b8d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=04298a5ae3de4c6ea19be2c49048b771)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:14.722600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8d8cf-1f5b-4171-8a76-4b70061b8d99 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:14.724662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 592/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Mon Sep 19 11:46:14 2022] GET /identity/v3/domains/04298a5ae3de4c6ea19be2c49048b771 => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:14.730582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be22816-b2de-4d8e-822c-13f568a69af9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:14.840912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be22816-b2de-4d8e-822c-13f568a69af9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:14.875989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3699a-b3e4-4cd8-86e5-63850d103b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:14.876597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3699a-b3e4-4cd8-86e5-63850d103b54 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:14.876877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3699a-b3e4-4cd8-86e5-63850d103b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:14.877146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3699a-b3e4-4cd8-86e5-63850d103b54 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:14.877896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3699a-b3e4-4cd8-86e5-63850d103b54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:14.879324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3699a-b3e4-4cd8-86e5-63850d103b54 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:14.908975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 581/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:46:14 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:46:17.187160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891f698-b11a-494b-8402-6e61aa8f3ece None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:17.187747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891f698-b11a-494b-8402-6e61aa8f3ece None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:17.187747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891f698-b11a-494b-8402-6e61aa8f3ece None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:17.189010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 593/1174] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:17.194709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9fba4-d279-4ae3-a787-a49c7db1f5e7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:17.195216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9fba4-d279-4ae3-a787-a49c7db1f5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:17.195358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9fba4-d279-4ae3-a787-a49c7db1f5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:17.214596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-33a9fba4-d279-4ae3-a787-a49c7db1f5e7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:17.215960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-33a9fba4-d279-4ae3-a787-a49c7db1f5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:17.235593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:17.289454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 582/1175] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:17.565532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891f698-b11a-494b-8402-6e61aa8f3ece None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:17.664279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891f698-b11a-494b-8402-6e61aa8f3ece None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:17.697607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83baf213-55f2-4e38-8b0e-dda4081fb53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:17.698262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baf213-55f2-4e38-8b0e-dda4081fb53a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:17.698543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baf213-55f2-4e38-8b0e-dda4081fb53a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:17.698817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baf213-55f2-4e38-8b0e-dda4081fb53a None admin] PATH_INFO: `/v3/domains/04298a5ae3de4c6ea19be2c49048b771` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:17.699932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83baf213-55f2-4e38-8b0e-dda4081fb53a None admin] RBAC: Authorizing `identity:get_domain(domain_id=04298a5ae3de4c6ea19be2c49048b771)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:17.701347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83baf213-55f2-4e38-8b0e-dda4081fb53a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:17.702956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 594/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Mon Sep 19 11:46:17 2022] GET /identity/v3/domains/04298a5ae3de4c6ea19be2c49048b771 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:17.709033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9fba4-d279-4ae3-a787-a49c7db1f5e7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:17.805981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9fba4-d279-4ae3-a787-a49c7db1f5e7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:17.839056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e153e9-0319-4a34-84d9-3a15b94aa1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:17.839586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e153e9-0319-4a34-84d9-3a15b94aa1c7 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:17.839865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e153e9-0319-4a34-84d9-3a15b94aa1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:17.840134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e153e9-0319-4a34-84d9-3a15b94aa1c7 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:17.840788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e153e9-0319-4a34-84d9-3a15b94aa1c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:17.842232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e153e9-0319-4a34-84d9-3a15b94aa1c7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:17.864519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 583/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:46:17 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:46:20.069902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc571a08-96f1-49fa-8a1e-b1881c3e7a2a None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:20.070432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc571a08-96f1-49fa-8a1e-b1881c3e7a2a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:20.070432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc571a08-96f1-49fa-8a1e-b1881c3e7a2a None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:20.071800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 595/1178] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:20.077191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52d72a-2592-45b7-a77f-61baf2e6c6ce None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:20.077603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52d72a-2592-45b7-a77f-61baf2e6c6ce None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:20.077730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52d72a-2592-45b7-a77f-61baf2e6c6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:20.096408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6e52d72a-2592-45b7-a77f-61baf2e6c6ce None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:20.097584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6e52d72a-2592-45b7-a77f-61baf2e6c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:20.100834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:20.164392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 584/1179] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:20.446129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc571a08-96f1-49fa-8a1e-b1881c3e7a2a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:20.467948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:20.544452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc571a08-96f1-49fa-8a1e-b1881c3e7a2a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:20.576958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:20.577561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:20.577875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:20.578393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:20.588612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:46:20.593457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:46:20.594612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:46:20.595654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 596/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:46:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:46:20.601159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52d72a-2592-45b7-a77f-61baf2e6c6ce None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:20.727692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52d72a-2592-45b7-a77f-61baf2e6c6ce None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:20.762637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:20.763245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:20.763528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:20.763802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:20.764472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:20.764778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:20.766178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:20.774315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 585/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:46:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:20.780279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:20.861003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:20.880485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9c074-95f9-4125-b0fc-9f6a61fd7bf1 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:20.913939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:20.914602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:20.914963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:20.915319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] PATH_INFO: `/v3/users/f4cb1db860d9456b9ef5560370af9f83` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:20.929050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] RBAC: Authorizing `identity:get_user(user_id=f4cb1db860d9456b9ef5560370af9f83)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:20.930514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:20.932136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 597/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:46:20 2022] GET /identity/v3/users/f4cb1db860d9456b9ef5560370af9f83 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:20.938357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:20.941181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:21.039027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bc77b-9486-4025-ab59-c9b0b7f8c2e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:21.075352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:21.075957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:21.076240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:21.076513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:21.143085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:21.144541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:21.146462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:21.242604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 586/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:46:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:46:21.248599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:21.346233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a264-4af9-406f-88be-a47281bbda37 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:21.380398 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893bb30-d03d-4cfa-bd6b-a15b480169a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:21.381534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893bb30-d03d-4cfa-bd6b-a15b480169a8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:21.381851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893bb30-d03d-4cfa-bd6b-a15b480169a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:21.382128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893bb30-d03d-4cfa-bd6b-a15b480169a8 None admin] PATH_INFO: `/v3/projects/04303e09e374463c8c8fb585e46d17a1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:21.383262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893bb30-d03d-4cfa-bd6b-a15b480169a8 None admin] RBAC: Authorizing `identity:get_project(project_id=04303e09e374463c8c8fb585e46d17a1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:21.384684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893bb30-d03d-4cfa-bd6b-a15b480169a8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:21.386348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 598/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:46:21 2022] GET /identity/v3/projects/04303e09e374463c8c8fb585e46d17a1 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:21.392576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:21.495460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7871f6-12a6-459a-a0a3-deda04e09584 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:21.498027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:21.532926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4e16e-9039-43a1-a762-4f24a3769cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:21.533553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4e16e-9039-43a1-a762-4f24a3769cba None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:21.533861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4e16e-9039-43a1-a762-4f24a3769cba None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:21.534131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4e16e-9039-43a1-a762-4f24a3769cba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:21.534882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4e16e-9039-43a1-a762-4f24a3769cba None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=04303e09e374463c8c8fb585e46d17a1, user.id=f4cb1db860d9456b9ef5560370af9f83) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:21.535189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4e16e-9039-43a1-a762-4f24a3769cba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:21.536600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4e16e-9039-43a1-a762-4f24a3769cba None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:21.542194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 587/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:46:21 2022] GET /identity/v3/role_assignments?user.id=f4cb1db860d9456b9ef5560370af9f83&scope.project.id=04303e09e374463c8c8fb585e46d17a1&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:23.720711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc2b9-7be2-4b33-a27b-1a3084624567 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:23.721281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc2b9-7be2-4b33-a27b-1a3084624567 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:23.721281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc2b9-7be2-4b33-a27b-1a3084624567 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:23.722636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 599/1186] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:23.728725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee306e3-1ac9-48c1-8f3b-3587888a1400 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:23.729246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee306e3-1ac9-48c1-8f3b-3587888a1400 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:23.729339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee306e3-1ac9-48c1-8f3b-3587888a1400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:23.748098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9ee306e3-1ac9-48c1-8f3b-3587888a1400 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:23.749314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ee306e3-1ac9-48c1-8f3b-3587888a1400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:23.821310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 588/1187] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:24.094941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dc2b9-7be2-4b33-a27b-1a3084624567 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:24.104612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:24.207454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dc2b9-7be2-4b33-a27b-1a3084624567 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:24.243306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:24.244144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:24.244425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:24.244696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:24.256745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:46:24.261950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:46:24.263179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:46:24.264568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 600/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:46:24.269447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee306e3-1ac9-48c1-8f3b-3587888a1400 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:24.367942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:24.374832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee306e3-1ac9-48c1-8f3b-3587888a1400 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:24.408745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:24.409361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:24.409695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:24.410016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:24.410689 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:24.411024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:24.413092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:24.422154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 589/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:24.428600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:24.507172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:24.543129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13db95-3ef4-4f74-8a23-02c51dd655c4 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:24.577498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:24.578414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:24.578697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:24.578969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] PATH_INFO: `/v3/users/f4cb1db860d9456b9ef5560370af9f83` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:24.580239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] RBAC: Authorizing `identity:get_user(user_id=f4cb1db860d9456b9ef5560370af9f83)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:24.581673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:24.583329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 601/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:46:24 2022] GET /identity/v3/users/f4cb1db860d9456b9ef5560370af9f83 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:24.589558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:24.689300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209e63a-0eb8-4570-8756-b74ad58beed3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:24.722314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a6884-4d09-4a8d-9058-5e0270a4f0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:24.722942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a6884-4d09-4a8d-9058-5e0270a4f0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:24.723216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a6884-4d09-4a8d-9058-5e0270a4f0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:24.723518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a6884-4d09-4a8d-9058-5e0270a4f0c5 None admin] PATH_INFO: `/v3/projects/04303e09e374463c8c8fb585e46d17a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:24.724633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a6884-4d09-4a8d-9058-5e0270a4f0c5 None admin] RBAC: Authorizing `identity:get_project(project_id=04303e09e374463c8c8fb585e46d17a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:24.726648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a6884-4d09-4a8d-9058-5e0270a4f0c5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:24.728276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 590/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:46:24 2022] GET /identity/v3/projects/04303e09e374463c8c8fb585e46d17a1 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:24.734663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:24.848668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a1968-34e0-4fb9-80e3-9264d5b6bc46 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:24.881927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71324c28-a62f-4df5-bdb2-464d97013c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:24.882578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324c28-a62f-4df5-bdb2-464d97013c39 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:24.882863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324c28-a62f-4df5-bdb2-464d97013c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:24.883138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324c28-a62f-4df5-bdb2-464d97013c39 None admin] PATH_INFO: `/v3/projects/04303e09e374463c8c8fb585e46d17a1/users/f4cb1db860d9456b9ef5560370af9f83/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:24.885077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71324c28-a62f-4df5-bdb2-464d97013c39 None admin] RBAC: Authorizing `identity:create_grant(project_id=04303e09e374463c8c8fb585e46d17a1, user_id=f4cb1db860d9456b9ef5560370af9f83, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:24.886525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71324c28-a62f-4df5-bdb2-464d97013c39 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:24.888741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:24.900685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 602/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:46:24 2022] PUT /identity/v3/projects/04303e09e374463c8c8fb585e46d17a1/users/f4cb1db860d9456b9ef5560370af9f83/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:46:27.071571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a7f65-4913-4a1c-9be5-8356457d35ca None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:27.072116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a7f65-4913-4a1c-9be5-8356457d35ca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:27.072239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a7f65-4913-4a1c-9be5-8356457d35ca None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:27.074590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 591/1193] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:46:27.080302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d01ebc-5313-42a2-b630-e9a626e09267 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:27.080795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d01ebc-5313-42a2-b630-e9a626e09267 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:27.080889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d01ebc-5313-42a2-b630-e9a626e09267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:27.099204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b2d01ebc-5313-42a2-b630-e9a626e09267 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:46:27.100442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b2d01ebc-5313-42a2-b630-e9a626e09267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:46:27.181708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 603/1194] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:46:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:46:27.447657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a7f65-4913-4a1c-9be5-8356457d35ca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:27.479609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:27.563373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a7f65-4913-4a1c-9be5-8356457d35ca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:27.601578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:27.602244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:27.602520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:27.602790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:27.613778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:46:27.619083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:46:27.620263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:46:27.621517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 592/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:46:27.626445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d01ebc-5313-42a2-b630-e9a626e09267 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:27.728779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d01ebc-5313-42a2-b630-e9a626e09267 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:27.754092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:27.766518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:27.767129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:27.767407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:27.767683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:27.768355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:27.768655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:27.770067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:27.780187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 604/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:27.786763 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:27.880279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:27.897100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc5bf7-49c0-4e83-bdfb-1512c02af97d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:27.934451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:27.935091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:27.935368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:27.935639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] PATH_INFO: `/v3/users/f4cb1db860d9456b9ef5560370af9f83` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:27.936841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] RBAC: Authorizing `identity:get_user(user_id=f4cb1db860d9456b9ef5560370af9f83)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:27.938337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:27.940002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 593/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Sep 19 11:46:27 2022] GET /identity/v3/users/f4cb1db860d9456b9ef5560370af9f83 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:46:27.946465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:28.050100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c1b65-3abc-4d29-b95e-2d43206d1d97 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:28.086490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:28.087217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:28.087358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:28.087630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:28.152565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:28.172002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:28.173440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:28.274916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 605/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:46:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:46:28.281219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:28.386448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e581d-ef7e-425f-912f-7db353fff229 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:28.423013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a2ef7-03ec-4b8b-80bb-f2de242de164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:28.423647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a2ef7-03ec-4b8b-80bb-f2de242de164 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:28.423926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a2ef7-03ec-4b8b-80bb-f2de242de164 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:28.424202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a2ef7-03ec-4b8b-80bb-f2de242de164 None admin] PATH_INFO: `/v3/projects/04303e09e374463c8c8fb585e46d17a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:28.425327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a2ef7-03ec-4b8b-80bb-f2de242de164 None admin] RBAC: Authorizing `identity:get_project(project_id=04303e09e374463c8c8fb585e46d17a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:28.426787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a2ef7-03ec-4b8b-80bb-f2de242de164 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:28.428981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 594/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Sep 19 11:46:28 2022] GET /identity/v3/projects/04303e09e374463c8c8fb585e46d17a1 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:28.435308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:28.522383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:28.547514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73e91-0d85-4f14-bf30-d6b2b0fbc161 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:28.582653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be78851-26e5-49c1-92c6-40dbbf7638d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:28.583249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78851-26e5-49c1-92c6-40dbbf7638d0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:28.583531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78851-26e5-49c1-92c6-40dbbf7638d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:28.584320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78851-26e5-49c1-92c6-40dbbf7638d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:28.585054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be78851-26e5-49c1-92c6-40dbbf7638d0 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=04303e09e374463c8c8fb585e46d17a1, user.id=f4cb1db860d9456b9ef5560370af9f83) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:46:28.585361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be78851-26e5-49c1-92c6-40dbbf7638d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:28.586807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be78851-26e5-49c1-92c6-40dbbf7638d0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:28.592957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 606/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:46:28 2022] GET /identity/v3/role_assignments?user.id=f4cb1db860d9456b9ef5560370af9f83&scope.project.id=04303e09e374463c8c8fb585e46d17a1&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:46:38.427694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:38.590041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=25 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:25.670050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b65a-c218-4a6f-939f-b4e3842e9543 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:25.670842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b65a-c218-4a6f-939f-b4e3842e9543 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:25.670842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b65a-c218-4a6f-939f-b4e3842e9543 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:25.672358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 595/1201] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:25.679430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71c67-d053-4729-9114-dff5fa0a6318 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:25.680004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71c67-d053-4729-9114-dff5fa0a6318 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:25.680004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71c67-d053-4729-9114-dff5fa0a6318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:25.703586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d5a71c67-d053-4729-9114-dff5fa0a6318 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:25.704923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d5a71c67-d053-4729-9114-dff5fa0a6318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:25.793945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 607/1202] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:47:26.091523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2b65a-c218-4a6f-939f-b4e3842e9543 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:26.229403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2b65a-c218-4a6f-939f-b4e3842e9543 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:26.275601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:26.276364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:26.276661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:26.276957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:26.291922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:26.293415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:26.314060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:26.315554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:26.316688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:26.316688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:26.316688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:26.316688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 596/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:26.322545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a71c67-d053-4729-9114-dff5fa0a6318 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:26.476824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a71c67-d053-4729-9114-dff5fa0a6318 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:26.514271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:26.523011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d295d8-adba-4a21-9737-18cff12be8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:26.523646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d295d8-adba-4a21-9737-18cff12be8bd None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:26.523897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d295d8-adba-4a21-9737-18cff12be8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:26.524185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d295d8-adba-4a21-9737-18cff12be8bd None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:26.524808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d295d8-adba-4a21-9737-18cff12be8bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:26.525219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d295d8-adba-4a21-9737-18cff12be8bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:26.526476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d295d8-adba-4a21-9737-18cff12be8bd None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:26.544532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 608/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:26.545425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:26.624359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:26.688803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6b5d5-7c73-423d-b3e7-107321335211 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:26.744134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb414b-8f5d-4f61-9b16-3b41b56876f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:26.745226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb414b-8f5d-4f61-9b16-3b41b56876f5 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:26.745598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb414b-8f5d-4f61-9b16-3b41b56876f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:26.746029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb414b-8f5d-4f61-9b16-3b41b56876f5 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:26.746820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb414b-8f5d-4f61-9b16-3b41b56876f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:26.748369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb414b-8f5d-4f61-9b16-3b41b56876f5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:26.775016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 597/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Sep 19 11:47:26 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:47:29.111143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48dad8-c990-404b-8798-2b43f9156d54 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:29.111688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48dad8-c990-404b-8798-2b43f9156d54 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:29.111776 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48dad8-c990-404b-8798-2b43f9156d54 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:29.113822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 609/1206] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:29.119497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8bda4-2e2c-4bf1-91ce-875a78c7a330 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:29.119938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8bda4-2e2c-4bf1-91ce-875a78c7a330 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:29.120059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8bda4-2e2c-4bf1-91ce-875a78c7a330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:29.142258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c9c8bda4-2e2c-4bf1-91ce-875a78c7a330 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:29.143474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c9c8bda4-2e2c-4bf1-91ce-875a78c7a330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:29.225981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 598/1207] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:47:29.530925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48dad8-c990-404b-8798-2b43f9156d54 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:29.662980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48dad8-c990-404b-8798-2b43f9156d54 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:29.703370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:29.704061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:29.704383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:29.704690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:29.719710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:47:29.725197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:29.726267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:29.727154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 610/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:47:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:29.734650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8bda4-2e2c-4bf1-91ce-875a78c7a330 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:29.785094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:29.856232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8bda4-2e2c-4bf1-91ce-875a78c7a330 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:29.899054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:29.899816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:29.900194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:29.900595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:29.901429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:29.901817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:29.903316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:29.914382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 599/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:47:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:29.920630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:29.981089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:30.046962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3342f0-68e9-434b-a6d3-2deb2f02c4b6 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:30.092238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:30.093039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:30.093391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:30.093702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:30.107479 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:30.109082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:30.127005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:30.131855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:30.132805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:30.132805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:30.132805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:30.132805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 611/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:30.135060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:30.255463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:30.259189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f8f4-2523-435f-82a7-6631f7c50900 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:30.311858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:30.312565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:30.312971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:30.313424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:30.314082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:30.314415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:30.316651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:30.330468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 600/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:30.337167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:30.449560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bf116-fdbb-42e7-8cd9-323e73077022 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:30.451507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:30.496126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:30.496774 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:30.497060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:30.497474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:30.504860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:30.506317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:47:30.518475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:30.519852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:30.520867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 612/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:30.525794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:30.654796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1413ae-9a77-4aaa-8a6a-105245f3e1d9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:30.695708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:30.696320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:30.696597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:30.696863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:30.697590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:30.697928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:30.699387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:30.721348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 601/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Sep 19 11:47:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:30.731068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:30.860002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5848532-5ec4-4987-a25b-2c8346d72595 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:30.900520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:30.901222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:30.901504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:30.901790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:30.934299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:30.976287 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:30.977887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:31.110382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 613/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:47:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:47:31.116966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:31.146898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:31.233786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836d2d-0e3a-4e27-b268-c4613708be7d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:31.269340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:31.269995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:31.270271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:31.270537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:31.281894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:31.283364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:31.300165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:31.306501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:31.309456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:31.309456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:31.309456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:31.309456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 602/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:31.309733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:31.378985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:31.426663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac66859-e899-43ec-b49d-ac89b09a29d5 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:31.466301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:31.466914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:31.467226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:31.467482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:31.468128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:31.468430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:31.469881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:31.490267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 614/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:31.492401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:31.575332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:31.613104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fa57c-60b0-4053-b562-f54e1ea120eb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:31.654826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:31.655584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:31.655989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:31.656474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:31.664396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:31.665990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:47:31.678296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:31.679496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:31.680669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:31.684551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 603/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:31.686235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:31.819622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34998d2-d7f6-4509-bba8-73724f717c20 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:31.851243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:31.867462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df8097-865b-47d0-a941-ad24016298c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:31.868182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df8097-865b-47d0-a941-ad24016298c0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:31.868535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df8097-865b-47d0-a941-ad24016298c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:31.868834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df8097-865b-47d0-a941-ad24016298c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:31.869745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df8097-865b-47d0-a941-ad24016298c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:31.869912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df8097-865b-47d0-a941-ad24016298c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:31.871332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df8097-865b-47d0-a941-ad24016298c0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:31.890873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 615/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:31.892161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:32.007598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeabf7b-d4d4-48c1-ad73-41bca6c833fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:32.042276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:32.050448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df157a76-424a-4b95-9b01-deab525f910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:32.051278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df157a76-424a-4b95-9b01-deab525f910f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:32.051630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df157a76-424a-4b95-9b01-deab525f910f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:32.051904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df157a76-424a-4b95-9b01-deab525f910f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:32.052622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df157a76-424a-4b95-9b01-deab525f910f None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=e38726440e2b444d9e2ccbb8681b9ffe) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:32.052927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df157a76-424a-4b95-9b01-deab525f910f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:32.054460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df157a76-424a-4b95-9b01-deab525f910f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:32.060135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 604/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:47:31 2022] GET /identity/v3/role_assignments?user.id=e38726440e2b444d9e2ccbb8681b9ffe&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:34.343673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5170-013b-4bf8-9f68-576ddc3fa15b None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:34.344232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5170-013b-4bf8-9f68-576ddc3fa15b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:34.344232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5170-013b-4bf8-9f68-576ddc3fa15b None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:34.345597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 616/1220] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:34.351699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf2d37-c2e3-4005-9a9b-ed7e7dd1f3e3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:34.352235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf2d37-c2e3-4005-9a9b-ed7e7dd1f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:34.352235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf2d37-c2e3-4005-9a9b-ed7e7dd1f3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:34.372289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ffcf2d37-c2e3-4005-9a9b-ed7e7dd1f3e3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:34.373503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ffcf2d37-c2e3-4005-9a9b-ed7e7dd1f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:34.451000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 605/1221] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:47:34.750368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5170-013b-4bf8-9f68-576ddc3fa15b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:34.866069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5170-013b-4bf8-9f68-576ddc3fa15b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:34.904812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:34.905544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:34.905919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:34.906196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:34.919059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:47:34.924380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:34.926456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:47:34.927718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:34.931965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 617/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:47:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:34.933015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf2d37-c2e3-4005-9a9b-ed7e7dd1f3e3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:35.048301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf2d37-c2e3-4005-9a9b-ed7e7dd1f3e3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:35.076237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:35.090906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:35.091522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:35.091872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:35.092168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:35.092918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:35.093216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:35.094612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:35.106417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 606/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:47:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:47:35.112906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:35.165344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:35.235400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d1442-5b9c-40f6-ad74-ad07f562acb0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:35.272531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:35.273208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:35.273510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:35.273810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:35.285490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:35.287067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:35.302882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:35.304125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:35.305074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:35.305074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:35.305074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:35.305074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 618/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:47:35.310362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:35.429100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aef178-09d2-4f0e-9f45-5c3f8e4801b9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:35.469286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:35.469947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:35.470224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:35.470494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:35.471135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:35.471439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:35.473417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:35.485437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 607/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:35.491231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:35.599396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:35.606093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed73cf-b761-4769-a873-3586933628ed None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:35.643330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:35.644569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:35.644882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:35.645154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:35.653376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:35.654870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:47:35.666165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:35.667458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:35.668535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:35.672261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 619/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:35.673315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:35.732103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:35.802629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003872c9-375a-4bcf-89df-ec8bceab0062 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:35.843310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:35.843982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:35.844250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:35.844551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:35.845303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:35.845633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:35.847783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:35.873233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 608/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Sep 19 11:47:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:47:35.874337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:35.992557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8233d5-884a-43b3-b366-5bf06dd3a244 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:36.031742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:36.032399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:36.032686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:36.032957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:36.044631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:36.046585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:36.062508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:36.069404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:36.070699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:36.070699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:36.070699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:36.070699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 620/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:36.070937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:36.189741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54a1cb-cce9-4083-a18b-5733421661a1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:36.193123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:36.229048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:36.230037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:36.230508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:36.230822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:36.231629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:36.232120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:36.233483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:36.247884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 609/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:36.254574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:36.264610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:36.371747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cfc26-385c-4b48-9a17-bc74842d59dc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:36.408737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:36.409811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:36.410158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:36.410425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:36.418113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:36.419784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:47:36.433677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:36.440199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:36.441388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 621/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:47:36.442064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:36.554201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9c0c-c9b1-4f04-81be-f550b15d1f75 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:36.590735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019d86c-d287-41a5-b370-ac3d60efa6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:36.591265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019d86c-d287-41a5-b370-ac3d60efa6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:36.591541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019d86c-d287-41a5-b370-ac3d60efa6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:36.591813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019d86c-d287-41a5-b370-ac3d60efa6f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:36.592521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019d86c-d287-41a5-b370-ac3d60efa6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:36.592801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019d86c-d287-41a5-b370-ac3d60efa6f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:36.594316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019d86c-d287-41a5-b370-ac3d60efa6f9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:36.608299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 610/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:36.614525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:36.686498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:36.727822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cda3c7-1915-4e85-8214-88cd64973a59 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:36.773388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf75df-f2c3-4739-81e4-7d2f333fb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:36.774436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf75df-f2c3-4739-81e4-7d2f333fb198 None admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:36.774854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf75df-f2c3-4739-81e4-7d2f333fb198 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:36.775315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf75df-f2c3-4739-81e4-7d2f333fb198 None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/e38726440e2b444d9e2ccbb8681b9ffe/roles/ce5c9f6a8b7c4522ad3a175ef1333656` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:36.796193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf75df-f2c3-4739-81e4-7d2f333fb198 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=e38726440e2b444d9e2ccbb8681b9ffe, role_id=ce5c9f6a8b7c4522ad3a175ef1333656)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:36.798661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf75df-f2c3-4739-81e4-7d2f333fb198 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:36.813630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 622/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:47:36 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/e38726440e2b444d9e2ccbb8681b9ffe/roles/ce5c9f6a8b7c4522ad3a175ef1333656 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:47:39.044505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b767-2a71-47c2-8c32-1d3b92e36fa8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:39.045099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b767-2a71-47c2-8c32-1d3b92e36fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:39.045099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b767-2a71-47c2-8c32-1d3b92e36fa8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:39.046398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 611/1233] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:39.052407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7889b9-1a7d-43f3-a77c-31542e2dc956 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:39.052893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7889b9-1a7d-43f3-a77c-31542e2dc956 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:39.052997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7889b9-1a7d-43f3-a77c-31542e2dc956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:39.073358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8c7889b9-1a7d-43f3-a77c-31542e2dc956 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:39.074593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8c7889b9-1a7d-43f3-a77c-31542e2dc956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:39.147982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 623/1234] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:47:39.413299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416b767-2a71-47c2-8c32-1d3b92e36fa8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:39.443551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:39.525317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416b767-2a71-47c2-8c32-1d3b92e36fa8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:39.562596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:39.563373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:39.563658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:39.563937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:39.578567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:47:39.585221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:39.586555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:39.587440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 612/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Sep 19 11:47:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:39.592878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7889b9-1a7d-43f3-a77c-31542e2dc956 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:39.646363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:39.706995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7889b9-1a7d-43f3-a77c-31542e2dc956 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:39.751886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:39.752858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:39.753295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:39.753833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:39.754867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] RBAC: Adding query filter params (name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:39.755495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:39.757995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:39.771123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 624/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Sep 19 11:47:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:39.777445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:39.895043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:39.915482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cf86f-3024-4e0b-abf4-2458d2e15518 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:39.955538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:39.956743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:39.957103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:39.957225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:39.969733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:39.971993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:39.989561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:39.990835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:39.991794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:39.991794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:39.991794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:39.991794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 613/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:39.997575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:40.105753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682110ff-e31b-4989-a003-ea97b8ff8842 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:40.115049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:40.141585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:40.142263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:40.142545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:40.142820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:40.144020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:40.144323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:40.145675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:40.159055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 625/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:40.165125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:40.280716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d5e16-d291-46ab-a8b3-8214f0c0f02d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:40.320405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:40.321030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:40.321304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:40.321578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:40.324237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:40.330315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:40.333232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:47:40.348637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:40.350934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:40.353052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 614/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:40.359138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:40.482992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29240324-8414-4489-8579-3563b5ea0735 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:40.521851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:40.522616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:40.522982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:40.523259 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:40.524023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:40.524332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:40.526022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:40.546835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 626/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Sep 19 11:47:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:47:40.554835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:40.673472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf40048-f6a6-4d9a-8604-4533d7dea53d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:40.709392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:40.710032 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:40.710497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:40.710623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:40.786081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:40.794577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:40.796257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:40.909955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 615/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:47:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:47:40.916539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:40.949630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:41.037113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4d897-e3da-4c49-9170-3084b5641bab None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:41.076291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:41.076919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:41.077197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:41.077468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:41.087728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:41.089231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:41.106404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:41.108239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:41.109464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:41.109464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:41.109464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:41.109464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 627/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:41.114720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:41.214773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:41.222414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56303d-67b2-4561-800e-d0e6c1a4826a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:41.261776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:41.262540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:41.262818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:41.263098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:41.263798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:41.264187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:41.265831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:41.279394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 616/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:41.285324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:41.372334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:41.396542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd87053-f577-43a5-bf93-2b830b77ee4a None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:41.433288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:41.433976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:41.434287 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:41.434586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:41.443481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:41.445237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:47:41.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:41.460146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:41.461307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 628/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:47:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:47:41.466309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:41.590109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c4d4d-cf43-4995-bc07-53210b5435b4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:41.649589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35017ca1-ecf9-4f10-8503-54aa3128a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:41.650659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017ca1-ecf9-4f10-8503-54aa3128a98a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:41.651087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017ca1-ecf9-4f10-8503-54aa3128a98a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:41.651564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017ca1-ecf9-4f10-8503-54aa3128a98a None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:41.652765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017ca1-ecf9-4f10-8503-54aa3128a98a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:41.653308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017ca1-ecf9-4f10-8503-54aa3128a98a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:41.655924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017ca1-ecf9-4f10-8503-54aa3128a98a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:41.672977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 617/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:47:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:47:41.679656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:41.807813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65cef3-af72-45d1-b28e-d93d73787fd6 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:41.851345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb09946-c4bd-4fe4-a747-7167a5cccb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:41.851966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09946-c4bd-4fe4-a747-7167a5cccb35 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:41.852238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09946-c4bd-4fe4-a747-7167a5cccb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:41.852510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09946-c4bd-4fe4-a747-7167a5cccb35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:41.853236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb09946-c4bd-4fe4-a747-7167a5cccb35 None admin] RBAC: Adding query filter params (role.id=ce5c9f6a8b7c4522ad3a175ef1333656, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=e38726440e2b444d9e2ccbb8681b9ffe) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:41.853584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb09946-c4bd-4fe4-a747-7167a5cccb35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:41.855018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb09946-c4bd-4fe4-a747-7167a5cccb35 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:41.861063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 629/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:47:41 2022] GET /identity/v3/role_assignments?user.id=e38726440e2b444d9e2ccbb8681b9ffe&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=ce5c9f6a8b7c4522ad3a175ef1333656 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:44.169564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703fd71-c96a-49dd-b678-fcb1fa595d2f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:44.170149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703fd71-c96a-49dd-b678-fcb1fa595d2f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:44.170149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703fd71-c96a-49dd-b678-fcb1fa595d2f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:44.171461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 618/1247] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:44.177089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2a08-701e-413b-b81b-4e2b19e67d73 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:44.177523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2a08-701e-413b-b81b-4e2b19e67d73 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:44.177635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2a08-701e-413b-b81b-4e2b19e67d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:44.184216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:44.198715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-cddd2a08-701e-413b-b81b-4e2b19e67d73 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:44.200525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cddd2a08-701e-413b-b81b-4e2b19e67d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:44.276452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 630/1248] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:47:44.554880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703fd71-c96a-49dd-b678-fcb1fa595d2f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:44.575454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:44.671029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703fd71-c96a-49dd-b678-fcb1fa595d2f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:44.706653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:44.707275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:44.707554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:44.707825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:44.718489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:47:44.723784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:44.724754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:44.725908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 619/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:47:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:44.731102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2a08-701e-413b-b81b-4e2b19e67d73 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:44.850121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2a08-701e-413b-b81b-4e2b19e67d73 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:44.883430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:44.889197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:44.889959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:44.890265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:44.890673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:44.891184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:44.891493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:44.892849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:44.901731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 631/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:47:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:44.907944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:44.996246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:45.024751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aa45c-1864-4eb4-a2ec-7150b0e0272c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:45.065995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:45.066638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:45.066916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:45.067180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:45.077997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:45.079443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:45.098744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:45.099907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:45.100895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:45.100895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:45.100895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:45.100895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 620/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:45.107959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:45.220329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ebe80-16db-400a-b769-72a614261ac8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:45.260102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:45.260710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:45.260969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:45.261243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:45.261908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:45.262209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:45.263743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:45.277228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 632/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:45.283499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:45.397534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412faa6-e75d-47f0-836c-299a9a39647f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:45.425327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:45.434009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:45.434715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:45.434999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:45.435272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:45.444594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:45.446066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:47:45.459817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:45.461163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:45.462527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:45.466395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 621/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:45.467412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:45.533369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:45.584780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019329b0-8a1e-46a7-be25-6c446ead0402 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:45.622318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:45.622931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:45.623211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:45.623485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:45.624216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:45.624525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:45.625965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:45.650280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 633/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Sep 19 11:47:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:45.651711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:45.768387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca5985-d51c-49e3-a900-c873360f08c5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:45.812599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:45.813297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:45.813659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:45.813998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:45.873841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:45.887284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:45.888758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:45.995000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 622/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:47:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:47:46.000905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:46.108984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e7248-0d75-4e92-87b9-532768346f24 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:46.123151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:46.147988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:46.148582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:46.148863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:46.149137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:46.167712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:46.169171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:46.184822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:46.185878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:46.186837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:46.186837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:46.186837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:46.186837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 634/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:46.192511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:46.285578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:46.300582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da0d9c-165b-4dd8-bf50-85cb64ed791e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:46.336473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:46.337086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:46.337367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:46.337640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:46.338312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:46.338616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:46.339969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:46.352510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 623/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:46.358700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:46.465640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9ab44-bfc3-4c10-bc7e-0023d72b189e None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:46.498582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:46.499202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:46.499475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:46.499740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:46.506369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:46.507779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:47:46.519268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:46.520397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:46.521551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 635/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:46.526490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:46.625526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae7740-f5ba-4346-9cb6-f625c06e4837 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:46.657836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379053e-0615-43ed-bc8e-ec7ae2d9b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:46.658434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379053e-0615-43ed-bc8e-ec7ae2d9b27e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:46.659255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379053e-0615-43ed-bc8e-ec7ae2d9b27e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:46.659526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379053e-0615-43ed-bc8e-ec7ae2d9b27e None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:46.660211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379053e-0615-43ed-bc8e-ec7ae2d9b27e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:46.660517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379053e-0615-43ed-bc8e-ec7ae2d9b27e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:46.662104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379053e-0615-43ed-bc8e-ec7ae2d9b27e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:46.673493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 624/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:46.682462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:46.710513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:46.792334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bda092-adbf-4a0f-9894-5de733806ef8 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:46.824690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab217c98-25c6-4a4e-ac98-523ec53864c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:46.825270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab217c98-25c6-4a4e-ac98-523ec53864c4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:46.825543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab217c98-25c6-4a4e-ac98-523ec53864c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:46.825836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab217c98-25c6-4a4e-ac98-523ec53864c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:46.826546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab217c98-25c6-4a4e-ac98-523ec53864c4 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=e38726440e2b444d9e2ccbb8681b9ffe) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:46.826849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab217c98-25c6-4a4e-ac98-523ec53864c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:46.828503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab217c98-25c6-4a4e-ac98-523ec53864c4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:46.834303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 636/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:47:46 2022] GET /identity/v3/role_assignments?user.id=e38726440e2b444d9e2ccbb8681b9ffe&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:48.973737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949076-31b2-44a6-81b6-5c7238f264f0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:48.974417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949076-31b2-44a6-81b6-5c7238f264f0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:48.974417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949076-31b2-44a6-81b6-5c7238f264f0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:48.975686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 625/1261] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:48.980883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638600e4-cf3a-490d-a308-5c907a72ed72 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:48.981310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638600e4-cf3a-490d-a308-5c907a72ed72 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:48.981406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638600e4-cf3a-490d-a308-5c907a72ed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:49.001341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-638600e4-cf3a-490d-a308-5c907a72ed72 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:49.003200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-638600e4-cf3a-490d-a308-5c907a72ed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:49.070820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 637/1262] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:47:49.342718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949076-31b2-44a6-81b6-5c7238f264f0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:49.353909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:49.445068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949076-31b2-44a6-81b6-5c7238f264f0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:49.478022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:49.478647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:49.478927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:49.479198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:49.489462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:47:49.494349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:49.495478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:49.496768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 626/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:49.501478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638600e4-cf3a-490d-a308-5c907a72ed72 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:49.519734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:49.599555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638600e4-cf3a-490d-a308-5c907a72ed72 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:49.633223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:49.633849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:49.634130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:49.634403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:49.635061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:49.635370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:49.636720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:49.644821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 638/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:49.650824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:49.718654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:49.757036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391a6e-68e2-485f-836d-7cafaa105c73 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:49.792279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:49.792906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:49.793184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:49.793454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:49.806706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:49.808148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:49.823754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:49.824973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:49.826142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:49.826142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:49.826142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:49.826142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 627/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:49.830984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:49.909164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:49.928144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02478-31af-4f75-906b-c38b8800317d None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:49.960279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:49.960871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:49.961147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:49.961418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:49.962079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:49.962381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:49.963726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:49.975480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 639/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:49.981227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:50.085546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41d13c-9d9e-47be-ac55-5a481eba49b4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:50.104801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:50.119068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:50.119694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:50.119972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:50.120244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:50.127506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:50.128927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:47:50.139944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:50.141140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:50.142615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 628/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:50.146916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:50.242508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d4968-cde0-4076-9ab0-bbc81d51dafc None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:50.274848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:50.275446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:50.275725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:50.275995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:50.276711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:50.277015 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:50.278449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:50.286860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:50.295160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 640/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Sep 19 11:47:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:50.304406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:50.409974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f9f55-d53b-4df2-ac3f-37aa7867d23e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:50.444361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:50.444974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:50.445249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:50.445520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:50.456894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:50.458430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:50.472396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:50.473624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:50.474593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:50.474593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:50.474593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:50.474593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 629/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:50.479483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:50.581908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6e59f-95ab-4399-8eca-6aa9ad35c9a7 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:50.616876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:50.617480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:50.617782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:50.618057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:50.618701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:50.619004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:50.620360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:50.631595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 641/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:50.637386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:50.670245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:50.742773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423af87-c492-47e2-97d7-19abf2ce5d76 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:50.787002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:50.787628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:50.787911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:50.788182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:50.795210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:50.796654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:47:50.808957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:50.810125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:50.811215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 630/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:50.816164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:50.875855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:50.917902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3816-65f9-4a6c-934d-0928f4513b77 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:50.956787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e1c43-018d-4849-b803-834656979eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:50.957393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e1c43-018d-4849-b803-834656979eef None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:50.957674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e1c43-018d-4849-b803-834656979eef None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:50.957976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e1c43-018d-4849-b803-834656979eef None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:50.958681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e1c43-018d-4849-b803-834656979eef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:50.958981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e1c43-018d-4849-b803-834656979eef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:50.960394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e1c43-018d-4849-b803-834656979eef None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:50.973200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 642/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:50.979291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:51.074742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:51.084985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559cab0-fe5c-4f42-9f7c-d9fd54e5fb27 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:51.119340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10fdf8-5105-438b-9367-59f2db76a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:51.120066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fdf8-5105-438b-9367-59f2db76a360 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:51.120346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fdf8-5105-438b-9367-59f2db76a360 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:51.120619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fdf8-5105-438b-9367-59f2db76a360 None admin] PATH_INFO: `/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/e38726440e2b444d9e2ccbb8681b9ffe/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:51.122600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10fdf8-5105-438b-9367-59f2db76a360 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb8af98c7bf40ca8e628a1d04083399, user_id=e38726440e2b444d9e2ccbb8681b9ffe, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:51.124121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10fdf8-5105-438b-9367-59f2db76a360 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:51.139369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 631/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Sep 19 11:47:50 2022] PUT /identity/v3/projects/fbb8af98c7bf40ca8e628a1d04083399/users/e38726440e2b444d9e2ccbb8681b9ffe/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:47:53.353217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571141f-0994-4c7b-a2e8-d391e2fbd1f7 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:53.353806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571141f-0994-4c7b-a2e8-d391e2fbd1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:53.353806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571141f-0994-4c7b-a2e8-d391e2fbd1f7 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:53.355100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 643/1274] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:53.360778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdd7f9-0088-40e7-9a71-cbb04837f1fc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:53.361830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdd7f9-0088-40e7-9a71-cbb04837f1fc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:53.361830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdd7f9-0088-40e7-9a71-cbb04837f1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:53.380920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e0fdd7f9-0088-40e7-9a71-cbb04837f1fc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:53.382203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e0fdd7f9-0088-40e7-9a71-cbb04837f1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:53.450884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 632/1275] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:47:53.718132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571141f-0994-4c7b-a2e8-d391e2fbd1f7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:53.856967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571141f-0994-4c7b-a2e8-d391e2fbd1f7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:53.875105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:53.892483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:53.893211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:53.893493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:53.893782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:53.903857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:47:53.909183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:53.913808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:53.914953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 644/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Sep 19 11:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:53.916425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdd7f9-0088-40e7-9a71-cbb04837f1fc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:53.999931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:54.019722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdd7f9-0088-40e7-9a71-cbb04837f1fc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:54.055484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:54.056093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:54.056373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:54.056647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:54.057321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:54.057634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:54.059036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:54.067799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 633/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Sep 19 11:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:54.073849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:54.193260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db662e-b6c2-41d0-aa73-9694c1f92d37 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:54.229626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:54.230294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:54.230575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:54.230849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:54.241369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:54.242842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:54.257738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:54.259083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:54.260308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:54.260308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:54.260308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:54.260308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 645/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:54.264870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:54.377911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce39f-e19d-471e-aaae-abe915413a01 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:54.414117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:54.414724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:54.415004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:54.415279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:54.415935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:54.416242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:54.417625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:54.431224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 634/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:47:54.437447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:54.471631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:54.544007 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138795-a71b-4669-97b6-f66285bc7e23 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:54.581624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:54.582287 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:54.582563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:54.582832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:54.590911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:54.592350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:47:54.605056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:54.606818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:54.609050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 646/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:54.613889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:54.618029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:54.729551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81479-f9bc-45ac-ac1a-edc2b7e0b0e0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:54.776380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:54.776985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:54.777260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:54.777530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:54.778288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:54.778593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:54.780048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:54.798418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 635/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Sep 19 11:47:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:54.804843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:54.901155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:54.911139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3319d87-ae7d-4512-8965-8018218631e6 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:54.949289 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:54.949946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:54.950234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:54.950496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:55.025577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:55.027033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:55.126156 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:55.139184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 647/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Sep 19 11:47:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:47:55.148408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:55.213407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:55.258605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06b21-2f4b-4bab-877f-800edb3c47cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:55.294744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:55.295374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:55.295655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:55.295924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:55.306668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:55.308123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:47:55.322992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:55.324207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:47:55.325507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:47:55.325507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:47:55.325507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:55.325507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 636/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:55.330331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:55.430198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f4ed-c636-428f-b04a-466f2d2842e0 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:55.470335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:55.471585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:55.471784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:55.472052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:55.472696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:55.472997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:55.474375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:55.486025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 648/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Sep 19 11:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:55.491802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:55.591257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bcae4-dfa4-484b-be57-d9daab1f2294 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:55.611788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:55.636610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:55.637265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:55.637542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:55.637836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:55.645747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:55.647220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:47:55.660027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:55.661189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 19 11:47:55.662456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 637/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Sep 19 11:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:55.667328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:55.721187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:55.775493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156713b-73bb-4a6d-a66e-00016405b222 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:55.833573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d085a60-75ba-4473-b1c2-60229f66be6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:55.834671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d085a60-75ba-4473-b1c2-60229f66be6f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:55.835179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d085a60-75ba-4473-b1c2-60229f66be6f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:55.835636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d085a60-75ba-4473-b1c2-60229f66be6f None admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:55.836345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d085a60-75ba-4473-b1c2-60229f66be6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:55.836654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d085a60-75ba-4473-b1c2-60229f66be6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:55.838945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d085a60-75ba-4473-b1c2-60229f66be6f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:55.854382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 649/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Sep 19 11:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:55.860639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:55.981014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c009be-eb49-4040-a786-e728b7fcfdb7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:56.015015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3b93c-1a4f-4c56-8abb-e41011e7d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:56.015616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b93c-1a4f-4c56-8abb-e41011e7d773 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:56.015895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b93c-1a4f-4c56-8abb-e41011e7d773 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:56.016168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b93c-1a4f-4c56-8abb-e41011e7d773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:56.016894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3b93c-1a4f-4c56-8abb-e41011e7d773 None admin] RBAC: Adding query filter params (role.id=0b2bdba44e3544f18a9d494b1acc8711, scope.project.id=fbb8af98c7bf40ca8e628a1d04083399, user.id=e38726440e2b444d9e2ccbb8681b9ffe) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:56.017200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3b93c-1a4f-4c56-8abb-e41011e7d773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:56.019276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3b93c-1a4f-4c56-8abb-e41011e7d773 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:56.025501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 638/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Sep 19 11:47:55 2022] GET /identity/v3/role_assignments?user.id=e38726440e2b444d9e2ccbb8681b9ffe&scope.project.id=fbb8af98c7bf40ca8e628a1d04083399&role.id=0b2bdba44e3544f18a9d494b1acc8711 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:58.243084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79c5f-549f-435a-9a27-0dc09ad7f56f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:58.243654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79c5f-549f-435a-9a27-0dc09ad7f56f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:58.243654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79c5f-549f-435a-9a27-0dc09ad7f56f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:58.244939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 650/1288] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:58.250135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e68d-1124-4de5-8398-9e817bd760ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:58.250598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e68d-1124-4de5-8398-9e817bd760ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:58.250713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e68d-1124-4de5-8398-9e817bd760ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:58.268605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cc50e68d-1124-4de5-8398-9e817bd760ad None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:58.269816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cc50e68d-1124-4de5-8398-9e817bd760ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:58.300300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:58.339985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 639/1289] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:47:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:47:58.606179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d79c5f-549f-435a-9a27-0dc09ad7f56f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:58.713231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d79c5f-549f-435a-9a27-0dc09ad7f56f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:58.725303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:58.751647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:58.752373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:58.752732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:58.753005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:58.753649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:58.755401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:47:58.768703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:47:58.770270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:47:58.771304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 651/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:47:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:47:58.776946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50e68d-1124-4de5-8398-9e817bd760ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:58.895553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50e68d-1124-4de5-8398-9e817bd760ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:58.932612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7679b-30fb-4ea1-8c08-b7aabd5a29d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:58.933227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7679b-30fb-4ea1-8c08-b7aabd5a29d2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:58.933510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7679b-30fb-4ea1-8c08-b7aabd5a29d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:58.933812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7679b-30fb-4ea1-8c08-b7aabd5a29d2 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:58.934459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7679b-30fb-4ea1-8c08-b7aabd5a29d2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:58.934770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7679b-30fb-4ea1-8c08-b7aabd5a29d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:58.936157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7679b-30fb-4ea1-8c08-b7aabd5a29d2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:58.941835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 640/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Sep 19 11:47:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:47:58.947664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:59.055999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130dafdc-cee6-4166-9e47-e78664e963b9 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:59.097049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb76b5-c439-4191-9859-6190ecda7d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:59.097671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb76b5-c439-4191-9859-6190ecda7d5e None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:59.098056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb76b5-c439-4191-9859-6190ecda7d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:59.098329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb76b5-c439-4191-9859-6190ecda7d5e None admin] PATH_INFO: `/v3/services` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:59.098970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb76b5-c439-4191-9859-6190ecda7d5e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:47:59.099278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb76b5-c439-4191-9859-6190ecda7d5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:59.100707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb76b5-c439-4191-9859-6190ecda7d5e None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:59.105827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 652/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Sep 19 11:47:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:48:01.274983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fda025-0d58-4888-99a5-b76119a7dafd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:01.275518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fda025-0d58-4888-99a5-b76119a7dafd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:01.275518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fda025-0d58-4888-99a5-b76119a7dafd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:01.276900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 641/1293] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:48:01.282342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60607eec-9952-40b1-9f93-ffc6a1741997 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:01.283316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60607eec-9952-40b1-9f93-ffc6a1741997 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:01.283419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60607eec-9952-40b1-9f93-ffc6a1741997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:01.306167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-60607eec-9952-40b1-9f93-ffc6a1741997 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:48:01.307584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-60607eec-9952-40b1-9f93-ffc6a1741997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:48:01.338694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:01.377618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 653/1294] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:48:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:48:01.648180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fda025-0d58-4888-99a5-b76119a7dafd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:01.679317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:01.762121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fda025-0d58-4888-99a5-b76119a7dafd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:01.811527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8cd3-7ce7-489e-8a9e-9081c373cc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:01.812388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8cd3-7ce7-489e-8a9e-9081c373cc6e None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:01.812756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8cd3-7ce7-489e-8a9e-9081c373cc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:01.813306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8cd3-7ce7-489e-8a9e-9081c373cc6e None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:01.815175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a8cd3-7ce7-489e-8a9e-9081c373cc6e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:01.817488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a8cd3-7ce7-489e-8a9e-9081c373cc6e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:01.836196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 642/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Sep 19 11:48:01 2022] POST /identity/v3/services => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:48:04.019820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2cf7-907d-42b1-b1aa-74ab1d82c40c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:04.020550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2cf7-907d-42b1-b1aa-74ab1d82c40c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:04.020550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2cf7-907d-42b1-b1aa-74ab1d82c40c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:04.021662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 654/1296] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:48:04.029383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff06b0-51a4-49a0-977e-4931df7349fb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:04.029950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff06b0-51a4-49a0-977e-4931df7349fb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:04.029950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff06b0-51a4-49a0-977e-4931df7349fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:04.049500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2dff06b0-51a4-49a0-977e-4931df7349fb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:48:04.050779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2dff06b0-51a4-49a0-977e-4931df7349fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:48:04.127845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 643/1297] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:48:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:48:04.396188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e2cf7-907d-42b1-b1aa-74ab1d82c40c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:04.540377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e2cf7-907d-42b1-b1aa-74ab1d82c40c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:04.580933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:04.581564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:04.581868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:04.582138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:04.582723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:04.584064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:48:04.593128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:48:04.595272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:48:04.597559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 655/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:48:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:48:04.602020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff06b0-51a4-49a0-977e-4931df7349fb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:04.623441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:04.724337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff06b0-51a4-49a0-977e-4931df7349fb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:04.763120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:04.763714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:04.764006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:04.764353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:04.765185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:48:04.765566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:04.767789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:04.777816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 644/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Sep 19 11:48:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:48:04.780819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:04.818424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:04.891433 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba89a58-e0f0-4896-8dfc-0f70bbae016f None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:04.925785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4c1bb-28e5-4110-9556-c9ccb14b23db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:04.926444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4c1bb-28e5-4110-9556-c9ccb14b23db None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:04.926729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4c1bb-28e5-4110-9556-c9ccb14b23db None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:04.927006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4c1bb-28e5-4110-9556-c9ccb14b23db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:04.927709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4c1bb-28e5-4110-9556-c9ccb14b23db None admin] RBAC: Adding query filter params (interface=public, service_id=326f1c8e4d7f49d5a7dac15f6d62fc8c, region_id=RegionOne) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:48:04.928021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4c1bb-28e5-4110-9556-c9ccb14b23db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:04.929377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4c1bb-28e5-4110-9556-c9ccb14b23db None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:04.934564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 656/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Sep 19 11:48:04 2022] GET /identity/v3/endpoints?service_id=326f1c8e4d7f49d5a7dac15f6d62fc8c&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:48:04.940864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:05.025675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:05.045108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a6053-900a-4749-9a2a-a293aa94ce8d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:05.080680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f62437-056a-4326-bf04-e7279c16933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:05.081289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f62437-056a-4326-bf04-e7279c16933a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:05.081567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f62437-056a-4326-bf04-e7279c16933a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:05.081872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f62437-056a-4326-bf04-e7279c16933a None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:05.082604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f62437-056a-4326-bf04-e7279c16933a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:05.083979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f62437-056a-4326-bf04-e7279c16933a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:05.090784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 645/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Sep 19 11:48:04 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:48:07.313702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0bbb-c241-4942-a5a4-f7cd79b52e0f None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:07.314748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0bbb-c241-4942-a5a4-f7cd79b52e0f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:07.314872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0bbb-c241-4942-a5a4-f7cd79b52e0f None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:07.316103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 657/1302] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:48:07.321594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e500f-9adf-44d8-978b-a713f52831e3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:07.322086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e500f-9adf-44d8-978b-a713f52831e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:07.322188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e500f-9adf-44d8-978b-a713f52831e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:07.342297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-263e500f-9adf-44d8-978b-a713f52831e3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:48:07.343498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-263e500f-9adf-44d8-978b-a713f52831e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:48:07.420482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 646/1303] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:48:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:48:07.687190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0bbb-c241-4942-a5a4-f7cd79b52e0f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:07.790194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:07.797127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0bbb-c241-4942-a5a4-f7cd79b52e0f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:07.832441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:07.833059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:07.833332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:07.833599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:07.834212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:07.835556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:48:07.844186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:48:07.846580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 11:48:07.848070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 658/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Sep 19 11:48:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:48:07.852186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e500f-9adf-44d8-978b-a713f52831e3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:07.930780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:07.954614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e500f-9adf-44d8-978b-a713f52831e3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:07.989423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939647-f1d3-4f67-a2b5-e344d852e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:07.990060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939647-f1d3-4f67-a2b5-e344d852e3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:07.990342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939647-f1d3-4f67-a2b5-e344d852e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:07.990617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939647-f1d3-4f67-a2b5-e344d852e3c9 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:07.991250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939647-f1d3-4f67-a2b5-e344d852e3c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:48:07.991557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939647-f1d3-4f67-a2b5-e344d852e3c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:07.992955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939647-f1d3-4f67-a2b5-e344d852e3c9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:07.998215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 647/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Sep 19 11:48:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:48:08.004363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:08.104295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5726ae-79c1-4af9-80b0-71201cc7ffb4 None admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:08.139155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb403765-2f9f-4f6b-9310-1a90f72e7283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:08.139773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb403765-2f9f-4f6b-9310-1a90f72e7283 None admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:08.140055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb403765-2f9f-4f6b-9310-1a90f72e7283 None admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:08.140329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb403765-2f9f-4f6b-9310-1a90f72e7283 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:08.141002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb403765-2f9f-4f6b-9310-1a90f72e7283 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:08.142467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb403765-2f9f-4f6b-9310-1a90f72e7283 None admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:08.168041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 659/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:48:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:48:17.995082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=30 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:18.159871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:14.608149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf1020-0764-4d50-b71d-4896b8ad16ba None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:14.609869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf1020-0764-4d50-b71d-4896b8ad16ba None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:14.610863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf1020-0764-4d50-b71d-4896b8ad16ba None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:14.614737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 648/1307] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:14.620975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4722e-4b6f-4f06-a882-b4bee0dbfc59 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:14.621872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4722e-4b6f-4f06-a882-b4bee0dbfc59 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:14.622362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4722e-4b6f-4f06-a882-b4bee0dbfc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:14.745128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c6d4722e-4b6f-4f06-a882-b4bee0dbfc59 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:14.746830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c6d4722e-4b6f-4f06-a882-b4bee0dbfc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:14.870458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 660/1308] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:15.169936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb8c2f-6bb6-47bf-a687-d4112e6ba4e6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:15.170742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb8c2f-6bb6-47bf-a687-d4112e6ba4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:15.171201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb8c2f-6bb6-47bf-a687-d4112e6ba4e6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:15.172555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 649/1309] 10.222.0.88 () {54 vars in 963 bytes} [Mon Sep 19 11:49:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:15.179333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7972e-ed43-4f73-8f96-2cb46aca0a79 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:15.179871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7972e-ed43-4f73-8f96-2cb46aca0a79 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:15.180146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7972e-ed43-4f73-8f96-2cb46aca0a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:15.276738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5ba7972e-ed43-4f73-8f96-2cb46aca0a79 None None] MFA Rules not processed for user `dee035c97cb6447382365faf6b6aca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:15.278217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5ba7972e-ed43-4f73-8f96-2cb46aca0a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:15.306210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:15.377646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 661/1310] 10.222.0.88 () {58 vars in 1060 bytes} [Mon Sep 19 11:49:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:15.386729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb8c2f-6bb6-47bf-a687-d4112e6ba4e6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:15.388620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5abb8c2f-6bb6-47bf-a687-d4112e6ba4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:15.503268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb8c2f-6bb6-47bf-a687-d4112e6ba4e6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:15.582563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4f1a8-2201-41d7-ac38-fed00db4a683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:15.583988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4f1a8-2201-41d7-ac38-fed00db4a683 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:15.584464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4f1a8-2201-41d7-ac38-fed00db4a683 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:15.584932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4f1a8-2201-41d7-ac38-fed00db4a683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:15.587575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9aa4f1a8-2201-41d7-ac38-fed00db4a683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:15.654244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:15.714630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4f1a8-2201-41d7-ac38-fed00db4a683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:15.716265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4f1a8-2201-41d7-ac38-fed00db4a683 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:15.736176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 650/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:18.187877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3feb8-ee4f-44a4-976f-25579a3903a3 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:18.188367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3feb8-ee4f-44a4-976f-25579a3903a3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:18.188367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3feb8-ee4f-44a4-976f-25579a3903a3 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:18.189712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 662/1312] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:18.195125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f6117-2e7d-433d-9960-125db32f61d0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:18.195531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f6117-2e7d-433d-9960-125db32f61d0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:18.195682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f6117-2e7d-433d-9960-125db32f61d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:18.265895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-447f6117-2e7d-433d-9960-125db32f61d0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:18.267112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-447f6117-2e7d-433d-9960-125db32f61d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:18.365161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 651/1313] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:18.646023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3feb8-ee4f-44a4-976f-25579a3903a3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:18.647590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-06b3feb8-ee4f-44a4-976f-25579a3903a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:18.756019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3feb8-ee4f-44a4-976f-25579a3903a3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:18.830555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:18.833290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7d24f-68c7-4618-9a00-3ec3aa2f6d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:18.833933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7d24f-68c7-4618-9a00-3ec3aa2f6d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:18.834559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7d24f-68c7-4618-9a00-3ec3aa2f6d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:18.834891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7d24f-68c7-4618-9a00-3ec3aa2f6d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:18.836867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a1b7d24f-68c7-4618-9a00-3ec3aa2f6d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:18.948253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7d24f-68c7-4618-9a00-3ec3aa2f6d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:18.950067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7d24f-68c7-4618-9a00-3ec3aa2f6d22 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:18.970401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 663/1314] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:21.459167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b14e9-5dcf-4baa-a2f5-d2e85df67537 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:21.459687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b14e9-5dcf-4baa-a2f5-d2e85df67537 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:21.459687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b14e9-5dcf-4baa-a2f5-d2e85df67537 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:21.460975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 652/1315] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:21.466337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6737587-dd07-454a-8137-e4375082ee44 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:21.466718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6737587-dd07-454a-8137-e4375082ee44 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:21.466883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6737587-dd07-454a-8137-e4375082ee44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:21.531943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c6737587-dd07-454a-8137-e4375082ee44 None None] MFA Rules not processed for user `d20c9d80453a47e9990b3a60cdaa40ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:21.533149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c6737587-dd07-454a-8137-e4375082ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:21.589834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:21.633190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 664/1316] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:21.918206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b14e9-5dcf-4baa-a2f5-d2e85df67537 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:21.919836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a27b14e9-5dcf-4baa-a2f5-d2e85df67537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:21.929730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:22.027660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b14e9-5dcf-4baa-a2f5-d2e85df67537 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:22.100353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ad640-5f49-4890-8cf6-89ea86ec161b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:22.100948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad640-5f49-4890-8cf6-89ea86ec161b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:22.101297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad640-5f49-4890-8cf6-89ea86ec161b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:22.101498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad640-5f49-4890-8cf6-89ea86ec161b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:22.103158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-961ad640-5f49-4890-8cf6-89ea86ec161b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:22.164304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:22.211318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ad640-5f49-4890-8cf6-89ea86ec161b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:22.212743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ad640-5f49-4890-8cf6-89ea86ec161b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:22.230193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 653/1317] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:24.933283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a87e61-cc95-4b59-b575-a814d11d09c7 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:24.933852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a87e61-cc95-4b59-b575-a814d11d09c7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:24.933852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a87e61-cc95-4b59-b575-a814d11d09c7 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:24.935219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 665/1318] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:24.940552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17c5b2-d616-4002-9fcc-1413d9e9d652 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:24.940965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17c5b2-d616-4002-9fcc-1413d9e9d652 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:24.941091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17c5b2-d616-4002-9fcc-1413d9e9d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:25.007079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cb17c5b2-d616-4002-9fcc-1413d9e9d652 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:25.008294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb17c5b2-d616-4002-9fcc-1413d9e9d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:25.106260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 654/1319] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:25.385919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a87e61-cc95-4b59-b575-a814d11d09c7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:25.387478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e7a87e61-cc95-4b59-b575-a814d11d09c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:25.492786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a87e61-cc95-4b59-b575-a814d11d09c7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:25.563525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96860f8c-5643-40be-bf7d-de1e97d8a868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:25.564122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96860f8c-5643-40be-bf7d-de1e97d8a868 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:25.564402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96860f8c-5643-40be-bf7d-de1e97d8a868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:25.564672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96860f8c-5643-40be-bf7d-de1e97d8a868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:25.566284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-96860f8c-5643-40be-bf7d-de1e97d8a868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:25.573916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:25.677624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96860f8c-5643-40be-bf7d-de1e97d8a868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:25.679495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96860f8c-5643-40be-bf7d-de1e97d8a868 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:25.701391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 666/1320] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:28.537053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbb53e-8fc4-468a-81d2-f457d87fd6fd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:28.537591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbb53e-8fc4-468a-81d2-f457d87fd6fd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:28.537591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbb53e-8fc4-468a-81d2-f457d87fd6fd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:28.539085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 655/1321] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:28.544524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67ca4-583a-4b32-a097-f26f1e0f5ba4 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:28.544936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67ca4-583a-4b32-a097-f26f1e0f5ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:28.545083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67ca4-583a-4b32-a097-f26f1e0f5ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:28.615677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-dfa67ca4-583a-4b32-a097-f26f1e0f5ba4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:28.617067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-dfa67ca4-583a-4b32-a097-f26f1e0f5ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:28.682880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:28.727319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 667/1322] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:29.034887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d667-5bb5-4bcd-85ee-92e6469c4764 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:29.035148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d667-5bb5-4bcd-85ee-92e6469c4764 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:29.035373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d667-5bb5-4bcd-85ee-92e6469c4764 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:29.036712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 656/1323] 10.222.0.88 () {54 vars in 963 bytes} [Mon Sep 19 11:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:49:29.043010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e20d-4d5e-41d5-a64b-704cf8d5d4cf None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:29.043240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e20d-4d5e-41d5-a64b-704cf8d5d4cf None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:29.043532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e20d-4d5e-41d5-a64b-704cf8d5d4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:29.123349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-50b9e20d-4d5e-41d5-a64b-704cf8d5d4cf None None] MFA Rules not processed for user `dee035c97cb6447382365faf6b6aca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:29.124544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-50b9e20d-4d5e-41d5-a64b-704cf8d5d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:29.204500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 668/1324] 10.222.0.88 () {58 vars in 1060 bytes} [Mon Sep 19 11:49:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:29.213574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15d667-5bb5-4bcd-85ee-92e6469c4764 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:29.215131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c15d667-5bb5-4bcd-85ee-92e6469c4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:29.235956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:29.321143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15d667-5bb5-4bcd-85ee-92e6469c4764 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:29.398710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47035e8e-3417-4f02-a4b8-8d439ff3b85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:29.399307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47035e8e-3417-4f02-a4b8-8d439ff3b85d service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:29.399578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47035e8e-3417-4f02-a4b8-8d439ff3b85d service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:29.399843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47035e8e-3417-4f02-a4b8-8d439ff3b85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:29.401406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-47035e8e-3417-4f02-a4b8-8d439ff3b85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:29.471747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:29.509056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47035e8e-3417-4f02-a4b8-8d439ff3b85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:29.510504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47035e8e-3417-4f02-a4b8-8d439ff3b85d service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:29.527764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 657/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:31.998472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a43db6-07a4-48f7-b153-ae8693eab8d7 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:31.998986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a43db6-07a4-48f7-b153-ae8693eab8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:31.998986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a43db6-07a4-48f7-b153-ae8693eab8d7 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:32.001630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 669/1326] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:32.005469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60f21e-de27-4754-9f84-bbf0d548b0f2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:32.005936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60f21e-de27-4754-9f84-bbf0d548b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:32.006202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60f21e-de27-4754-9f84-bbf0d548b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:32.074095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3b60f21e-de27-4754-9f84-bbf0d548b0f2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:32.075369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3b60f21e-de27-4754-9f84-bbf0d548b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:32.175762 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 658/1327] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:32.457224 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a43db6-07a4-48f7-b153-ae8693eab8d7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:32.458826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-65a43db6-07a4-48f7-b153-ae8693eab8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:32.519047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:32.563429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a43db6-07a4-48f7-b153-ae8693eab8d7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:32.637367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d6e66-30fa-4dc5-8707-ff4880b0b5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:32.638008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d6e66-30fa-4dc5-8707-ff4880b0b5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:32.638283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d6e66-30fa-4dc5-8707-ff4880b0b5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:32.638547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d6e66-30fa-4dc5-8707-ff4880b0b5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:32.640125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cb8d6e66-30fa-4dc5-8707-ff4880b0b5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:32.747734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:32.750338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d6e66-30fa-4dc5-8707-ff4880b0b5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:32.752390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d6e66-30fa-4dc5-8707-ff4880b0b5b2 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:32.770982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 670/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:35.020203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57dae-2cb1-49f2-a9a8-49f3c4164fb1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:35.020792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57dae-2cb1-49f2-a9a8-49f3c4164fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:35.020792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57dae-2cb1-49f2-a9a8-49f3c4164fb1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:35.022196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 659/1329] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:35.029408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526a1c-7c6f-4961-bf79-11d88a18ce37 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:35.029956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526a1c-7c6f-4961-bf79-11d88a18ce37 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:35.030072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526a1c-7c6f-4961-bf79-11d88a18ce37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:35.107476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-40526a1c-7c6f-4961-bf79-11d88a18ce37 None None] MFA Rules not processed for user `d20c9d80453a47e9990b3a60cdaa40ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:35.108762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-40526a1c-7c6f-4961-bf79-11d88a18ce37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:35.216711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 671/1330] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:35.500263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57dae-2cb1-49f2-a9a8-49f3c4164fb1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:35.501784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-88c57dae-2cb1-49f2-a9a8-49f3c4164fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:35.534057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:35.610421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57dae-2cb1-49f2-a9a8-49f3c4164fb1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:35.690659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ec138-2f5b-4653-994f-757c8e5efb15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:35.691265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ec138-2f5b-4653-994f-757c8e5efb15 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:35.691540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ec138-2f5b-4653-994f-757c8e5efb15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:35.691810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ec138-2f5b-4653-994f-757c8e5efb15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:35.693385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b49ec138-2f5b-4653-994f-757c8e5efb15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:35.776456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:35.809084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ec138-2f5b-4653-994f-757c8e5efb15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:35.811001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ec138-2f5b-4653-994f-757c8e5efb15 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:35.829625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 660/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:38.237381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a847cfd-e124-4a9c-973d-f2d35f822146 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:38.237974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a847cfd-e124-4a9c-973d-f2d35f822146 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:38.237974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a847cfd-e124-4a9c-973d-f2d35f822146 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:38.239187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 672/1332] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:38.244546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21b7c2-b475-411b-beca-8282d1442e0e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:38.245010 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21b7c2-b475-411b-beca-8282d1442e0e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:38.245090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21b7c2-b475-411b-beca-8282d1442e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:38.312587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4b21b7c2-b475-411b-beca-8282d1442e0e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:38.313829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b21b7c2-b475-411b-beca-8282d1442e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:38.415660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 661/1333] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:38.702330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a847cfd-e124-4a9c-973d-f2d35f822146 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:38.703883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4a847cfd-e124-4a9c-973d-f2d35f822146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:38.776349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:38.819735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a847cfd-e124-4a9c-973d-f2d35f822146 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:38.899471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773992c4-5358-4bca-be6f-930574aedc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:38.900078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773992c4-5358-4bca-be6f-930574aedc83 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:38.900356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773992c4-5358-4bca-be6f-930574aedc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:38.900629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773992c4-5358-4bca-be6f-930574aedc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:38.902307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-773992c4-5358-4bca-be6f-930574aedc83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:39.019923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773992c4-5358-4bca-be6f-930574aedc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:39.021352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773992c4-5358-4bca-be6f-930574aedc83 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:39.032133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:39.040892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 673/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:41.296564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3894559-a662-4392-a1ee-21b5acc3b475 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:41.297170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3894559-a662-4392-a1ee-21b5acc3b475 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:41.297170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3894559-a662-4392-a1ee-21b5acc3b475 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:41.298454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 662/1335] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:41.303864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89debf3d-d4fd-4ef8-b679-da581d587c34 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:41.304303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89debf3d-d4fd-4ef8-b679-da581d587c34 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:41.304411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89debf3d-d4fd-4ef8-b679-da581d587c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:41.372546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-89debf3d-d4fd-4ef8-b679-da581d587c34 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:41.373784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-89debf3d-d4fd-4ef8-b679-da581d587c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:41.469455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 674/1336] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:41.760544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3894559-a662-4392-a1ee-21b5acc3b475 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:41.762109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f3894559-a662-4392-a1ee-21b5acc3b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:41.801935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:41.868162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3894559-a662-4392-a1ee-21b5acc3b475 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:41.943013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ab0b3-84f1-4108-ba1f-c28a00e205e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:41.943653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ab0b3-84f1-4108-ba1f-c28a00e205e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:41.943934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ab0b3-84f1-4108-ba1f-c28a00e205e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:41.944207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ab0b3-84f1-4108-ba1f-c28a00e205e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:41.945815 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a88ab0b3-84f1-4108-ba1f-c28a00e205e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:42.027490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:42.053380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ab0b3-84f1-4108-ba1f-c28a00e205e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:42.054857 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ab0b3-84f1-4108-ba1f-c28a00e205e4 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:42.073336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 663/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:44.602728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6e440-5976-4d02-bd13-8c5d16af6f9e None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:44.603286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6e440-5976-4d02-bd13-8c5d16af6f9e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:44.603286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6e440-5976-4d02-bd13-8c5d16af6f9e None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:44.604746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 675/1338] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:44.611112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa376422-8690-4b31-82db-4639eaccbd45 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:44.611624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa376422-8690-4b31-82db-4639eaccbd45 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:44.611758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa376422-8690-4b31-82db-4639eaccbd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:44.676617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fa376422-8690-4b31-82db-4639eaccbd45 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:44.677837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fa376422-8690-4b31-82db-4639eaccbd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:44.772895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 664/1339] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:45.058307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6e440-5976-4d02-bd13-8c5d16af6f9e None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:45.060450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fdb6e440-5976-4d02-bd13-8c5d16af6f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:45.137065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:45.164285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6e440-5976-4d02-bd13-8c5d16af6f9e None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:45.240175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113e065-2f55-45fd-8ce1-0ce95f422c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:45.240779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113e065-2f55-45fd-8ce1-0ce95f422c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:45.241068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113e065-2f55-45fd-8ce1-0ce95f422c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:45.241340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113e065-2f55-45fd-8ce1-0ce95f422c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:45.242968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b113e065-2f55-45fd-8ce1-0ce95f422c56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:45.352380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113e065-2f55-45fd-8ce1-0ce95f422c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:45.353725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113e065-2f55-45fd-8ce1-0ce95f422c56 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:45.372490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 676/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:49.492700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed69de-c0cf-4c03-a6a2-07fb53db3d9b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:49.493201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed69de-c0cf-4c03-a6a2-07fb53db3d9b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:49.493382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed69de-c0cf-4c03-a6a2-07fb53db3d9b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:49.494824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 665/1341] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:49.500665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaf442-094f-4b25-9d96-5af463778852 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:49.501214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaf442-094f-4b25-9d96-5af463778852 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:49.501214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaf442-094f-4b25-9d96-5af463778852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:49.506672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:49.577927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5faaf442-094f-4b25-9d96-5af463778852 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:49.579234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5faaf442-094f-4b25-9d96-5af463778852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:49.694165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 677/1342] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:49.975311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed69de-c0cf-4c03-a6a2-07fb53db3d9b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:49.976840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0aed69de-c0cf-4c03-a6a2-07fb53db3d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:50.024929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:50.082383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed69de-c0cf-4c03-a6a2-07fb53db3d9b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:50.157784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92232-90ab-48b9-a791-171ea39d8df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:50.158408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92232-90ab-48b9-a791-171ea39d8df9 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:50.158685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92232-90ab-48b9-a791-171ea39d8df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:50.158954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92232-90ab-48b9-a791-171ea39d8df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:50.160522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f1f92232-90ab-48b9-a791-171ea39d8df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:50.254170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:50.271257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92232-90ab-48b9-a791-171ea39d8df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:50.272695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92232-90ab-48b9-a791-171ea39d8df9 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:50.292257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 666/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:53.147087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438567c-e22f-4145-8b83-c8f4a86daeb1 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:53.147087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438567c-e22f-4145-8b83-c8f4a86daeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:53.147087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438567c-e22f-4145-8b83-c8f4a86daeb1 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:53.148549 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 678/1344] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:53.153634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4b1c-6664-47d3-8f45-cdfe2f74e3e3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:53.154092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4b1c-6664-47d3-8f45-cdfe2f74e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:53.154226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4b1c-6664-47d3-8f45-cdfe2f74e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:53.221160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b48c4b1c-6664-47d3-8f45-cdfe2f74e3e3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:53.222376 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b48c4b1c-6664-47d3-8f45-cdfe2f74e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:53.330756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 667/1345] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:53.615963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438567c-e22f-4145-8b83-c8f4a86daeb1 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:53.617512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d438567c-e22f-4145-8b83-c8f4a86daeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:53.713531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:53.732257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438567c-e22f-4145-8b83-c8f4a86daeb1 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:53.818077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01047c-d349-4674-b3b8-4e2dda5c848b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:53.818610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01047c-d349-4674-b3b8-4e2dda5c848b service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:53.818912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01047c-d349-4674-b3b8-4e2dda5c848b service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:53.819148 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01047c-d349-4674-b3b8-4e2dda5c848b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:53.820763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fe01047c-d349-4674-b3b8-4e2dda5c848b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:53.940079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01047c-d349-4674-b3b8-4e2dda5c848b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:53.941512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01047c-d349-4674-b3b8-4e2dda5c848b service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:53.960184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 679/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:49:58.777181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1446-a7ee-426c-8494-0119721e2d95 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:58.778481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1446-a7ee-426c-8494-0119721e2d95 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:58.778481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1446-a7ee-426c-8494-0119721e2d95 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:58.779772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 668/1347] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:49:58.785282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843f6b-5bb6-40cf-8a24-068a1ad68ac2 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:58.785959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843f6b-5bb6-40cf-8a24-068a1ad68ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:58.785959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843f6b-5bb6-40cf-8a24-068a1ad68ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:58.802562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:58.869250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5c843f6b-5bb6-40cf-8a24-068a1ad68ac2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:49:58.870506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5c843f6b-5bb6-40cf-8a24-068a1ad68ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:58.970195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 680/1348] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:49:59.266122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d1446-a7ee-426c-8494-0119721e2d95 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:49:59.267663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-533d1446-a7ee-426c-8494-0119721e2d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:59.324360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:59.370724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d1446-a7ee-426c-8494-0119721e2d95 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:49:59.446488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84827c-0be3-4e3d-a651-bb0d91c457e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:49:59.447006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84827c-0be3-4e3d-a651-bb0d91c457e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:49:59.447282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84827c-0be3-4e3d-a651-bb0d91c457e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:49:59.447547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84827c-0be3-4e3d-a651-bb0d91c457e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:49:59.449127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de84827c-0be3-4e3d-a651-bb0d91c457e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:49:59.559540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:49:59.563428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84827c-0be3-4e3d-a651-bb0d91c457e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:49:59.565153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84827c-0be3-4e3d-a651-bb0d91c457e3 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:49:59.583745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 669/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:04.551512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebedf8-c9e4-4280-b036-a7eea6bf79c0 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:04.552091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebedf8-c9e4-4280-b036-a7eea6bf79c0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:04.552478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebedf8-c9e4-4280-b036-a7eea6bf79c0 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:04.553922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 681/1350] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:50:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:04.559024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f49fdb-297d-4902-accd-bf7a5d284592 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:04.559519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f49fdb-297d-4902-accd-bf7a5d284592 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:04.559612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f49fdb-297d-4902-accd-bf7a5d284592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:04.667068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a6f49fdb-297d-4902-accd-bf7a5d284592 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:04.668126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6f49fdb-297d-4902-accd-bf7a5d284592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:04.815561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 670/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:08.968672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=59 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:09.227110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfd01f-30a2-404a-838a-31ee77ffc111 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:09.227288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfd01f-30a2-404a-838a-31ee77ffc111 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:09.227573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfd01f-30a2-404a-838a-31ee77ffc111 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:09.229143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 682/1352] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:09.234233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844f1c-6f93-4f03-a5a2-f4d380402325 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:09.234740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844f1c-6f93-4f03-a5a2-f4d380402325 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:09.235007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844f1c-6f93-4f03-a5a2-f4d380402325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:09.291349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:09.316800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a7844f1c-6f93-4f03-a5a2-f4d380402325 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:09.318435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a7844f1c-6f93-4f03-a5a2-f4d380402325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:09.438460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 671/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:50:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:10.869962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5863b9-f77a-409b-84f7-96136e569072 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:10.870520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5863b9-f77a-409b-84f7-96136e569072 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:10.870520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5863b9-f77a-409b-84f7-96136e569072 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:10.871843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 683/1354] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:10.876995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f639d1-5563-4a1a-92cb-15bf973042c1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:10.877473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f639d1-5563-4a1a-92cb-15bf973042c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:10.877698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f639d1-5563-4a1a-92cb-15bf973042c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:10.968835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-17f639d1-5563-4a1a-92cb-15bf973042c1 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:10.970189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-17f639d1-5563-4a1a-92cb-15bf973042c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:11.012372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:11.081344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 672/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:50:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:14.708123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38db69-0464-44cd-b832-cc65babb71fe None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:14.708628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38db69-0464-44cd-b832-cc65babb71fe None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:14.708628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38db69-0464-44cd-b832-cc65babb71fe None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:14.711108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 684/1356] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:14.718877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0e638-ccf2-44a4-9189-13df6fa2ff3f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:14.719469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0e638-ccf2-44a4-9189-13df6fa2ff3f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:14.719469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0e638-ccf2-44a4-9189-13df6fa2ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:14.796660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-57b0e638-ccf2-44a4-9189-13df6fa2ff3f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:14.797796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-57b0e638-ccf2-44a4-9189-13df6fa2ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:14.907210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 673/1357] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:15.211709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fd0a6-5f52-456a-b546-5b8322825887 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:15.211996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fd0a6-5f52-456a-b546-5b8322825887 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:15.212270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fd0a6-5f52-456a-b546-5b8322825887 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:15.213570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 685/1358] 10.222.0.88 () {54 vars in 965 bytes} [Mon Sep 19 11:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:15.222973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193560f-0a57-42fd-946c-507da927446a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:15.223556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193560f-0a57-42fd-946c-507da927446a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:15.224014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193560f-0a57-42fd-946c-507da927446a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:15.249302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:15.315499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9193560f-0a57-42fd-946c-507da927446a None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:15.317042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9193560f-0a57-42fd-946c-507da927446a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:15.452508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 674/1359] 10.222.0.88 () {58 vars in 1062 bytes} [Mon Sep 19 11:50:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:50:15.462094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fd0a6-5f52-456a-b546-5b8322825887 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:15.463603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-769fd0a6-5f52-456a-b546-5b8322825887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:15.621244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fd0a6-5f52-456a-b546-5b8322825887 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:15.696413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:15.738336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b79b69-e6b4-4ac8-85bf-92102f646f5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:15.739038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79b69-e6b4-4ac8-85bf-92102f646f5d service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:15.739304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79b69-e6b4-4ac8-85bf-92102f646f5d service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:15.739477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79b69-e6b4-4ac8-85bf-92102f646f5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:15.741027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-18b79b69-e6b4-4ac8-85bf-92102f646f5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:15.837550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b79b69-e6b4-4ac8-85bf-92102f646f5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:15.839209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b79b69-e6b4-4ac8-85bf-92102f646f5d service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:15.861310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 686/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:18.294416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c5f6d-aa79-41eb-a657-4a04c1ed3718 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:18.294979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c5f6d-aa79-41eb-a657-4a04c1ed3718 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:18.294979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c5f6d-aa79-41eb-a657-4a04c1ed3718 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:18.296275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 675/1361] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:50:18.301684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3d079-b81a-422d-a624-f350b119b283 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:18.302164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3d079-b81a-422d-a624-f350b119b283 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:18.302473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3d079-b81a-422d-a624-f350b119b283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:18.407629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-52f3d079-b81a-422d-a624-f350b119b283 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:18.408911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-52f3d079-b81a-422d-a624-f350b119b283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:18.450168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:18.525690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 687/1362] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:18.821154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c5f6d-aa79-41eb-a657-4a04c1ed3718 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:18.822732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b87c5f6d-aa79-41eb-a657-4a04c1ed3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:18.896860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:19.006897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c5f6d-aa79-41eb-a657-4a04c1ed3718 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:19.152350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfd6a7-f012-48fc-b942-fb129d0a476b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:19.153068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd6a7-f012-48fc-b942-fb129d0a476b service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:19.153343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd6a7-f012-48fc-b942-fb129d0a476b service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:19.153614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd6a7-f012-48fc-b942-fb129d0a476b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:19.155269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b8dfd6a7-f012-48fc-b942-fb129d0a476b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:19.178372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:19.251101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfd6a7-f012-48fc-b942-fb129d0a476b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:19.252736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfd6a7-f012-48fc-b942-fb129d0a476b service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:19.273875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 676/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:21.816997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da060-7026-4503-8b1d-cf1e21415165 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:21.817537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da060-7026-4503-8b1d-cf1e21415165 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:21.817537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da060-7026-4503-8b1d-cf1e21415165 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:21.818919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 688/1364] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:21.824841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bc1d9-cfeb-4dec-ab98-97526c7cf76d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:21.825294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bc1d9-cfeb-4dec-ab98-97526c7cf76d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:21.825389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bc1d9-cfeb-4dec-ab98-97526c7cf76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:21.894990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-619bc1d9-cfeb-4dec-ab98-97526c7cf76d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:21.896199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-619bc1d9-cfeb-4dec-ab98-97526c7cf76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:22.004026 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:22.010116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 677/1365] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:22.303358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4156387-1bb1-40fb-bef8-2ceeece4ec6d None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:22.303682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4156387-1bb1-40fb-bef8-2ceeece4ec6d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:22.303925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4156387-1bb1-40fb-bef8-2ceeece4ec6d None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:22.305375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 689/1366] 10.222.0.88 () {54 vars in 965 bytes} [Mon Sep 19 11:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:22.311906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a984c-911b-4368-a7e2-9593210f1fd9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:22.312308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a984c-911b-4368-a7e2-9593210f1fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:22.312681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a984c-911b-4368-a7e2-9593210f1fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:22.403881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7c3a984c-911b-4368-a7e2-9593210f1fd9 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:22.405043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7c3a984c-911b-4368-a7e2-9593210f1fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:22.528940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 678/1367] 10.222.0.88 () {58 vars in 1062 bytes} [Mon Sep 19 11:50:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:22.536789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4156387-1bb1-40fb-bef8-2ceeece4ec6d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:22.538348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c4156387-1bb1-40fb-bef8-2ceeece4ec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:22.699669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4156387-1bb1-40fb-bef8-2ceeece4ec6d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:22.721788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:22.825426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc1f0-2d48-4584-8725-6bfadc52ded1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:22.826195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc1f0-2d48-4584-8725-6bfadc52ded1 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:22.826571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc1f0-2d48-4584-8725-6bfadc52ded1 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:22.826937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc1f0-2d48-4584-8725-6bfadc52ded1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:22.829003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6dafc1f0-2d48-4584-8725-6bfadc52ded1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:22.927426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc1f0-2d48-4584-8725-6bfadc52ded1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:22.929253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc1f0-2d48-4584-8725-6bfadc52ded1 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:22.952788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 690/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:23.055301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d7720-ad5f-41ca-912a-36c437fea5fb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:23.055780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d7720-ad5f-41ca-912a-36c437fea5fb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:23.055880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d7720-ad5f-41ca-912a-36c437fea5fb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:23.057199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 679/1369] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:23.063061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5733e-2037-4059-8068-5ab7ff652f3f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:23.063498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5733e-2037-4059-8068-5ab7ff652f3f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:23.063881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5733e-2037-4059-8068-5ab7ff652f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:23.104185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:23.167717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3ee5733e-2037-4059-8068-5ab7ff652f3f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:23.170340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3ee5733e-2037-4059-8068-5ab7ff652f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:23.284880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 691/1370] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:23.622608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a1c2e-8c83-4268-a8d4-d0b653ff2661 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:23.622608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a1c2e-8c83-4268-a8d4-d0b653ff2661 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:23.622608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a1c2e-8c83-4268-a8d4-d0b653ff2661 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:23.624864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 680/1371] 10.222.0.88 () {54 vars in 967 bytes} [Mon Sep 19 11:50:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:23.630711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bda887-5dc6-402c-ae2d-4e0250a8db62 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:23.631079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bda887-5dc6-402c-ae2d-4e0250a8db62 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:23.631422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bda887-5dc6-402c-ae2d-4e0250a8db62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:23.715001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b4bda887-5dc6-402c-ae2d-4e0250a8db62 None None] MFA Rules not processed for user `37099ac7c39948f0afd272a2b41949ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:23.716426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b4bda887-5dc6-402c-ae2d-4e0250a8db62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:23.770812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:23.805420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 692/1372] 10.222.0.88 () {58 vars in 1064 bytes} [Mon Sep 19 11:50:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:23.813778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a1c2e-8c83-4268-a8d4-d0b653ff2661 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:23.815330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a52a1c2e-8c83-4268-a8d4-d0b653ff2661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:23.874615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:23.936954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a1c2e-8c83-4268-a8d4-d0b653ff2661 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:24.036833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d67607-0466-451c-ab05-165eeee572e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:24.037514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d67607-0466-451c-ab05-165eeee572e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:24.037861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d67607-0466-451c-ab05-165eeee572e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:24.038068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d67607-0466-451c-ab05-165eeee572e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:24.039836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d6d67607-0466-451c-ab05-165eeee572e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:24.165357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:24.174702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d67607-0466-451c-ab05-165eeee572e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:24.175580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d67607-0466-451c-ab05-165eeee572e1 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:24.198075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 681/1373] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:50:25.208360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69054fab-d696-486a-a1cb-47cb27bb453a None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:25.209025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69054fab-d696-486a-a1cb-47cb27bb453a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:25.209025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69054fab-d696-486a-a1cb-47cb27bb453a None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:25.210389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 693/1374] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:50:25.216873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532bac6-5686-41f7-8ceb-3564f9aa60c7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:25.217321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532bac6-5686-41f7-8ceb-3564f9aa60c7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:25.217416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532bac6-5686-41f7-8ceb-3564f9aa60c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:25.283837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1532bac6-5686-41f7-8ceb-3564f9aa60c7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:25.285045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1532bac6-5686-41f7-8ceb-3564f9aa60c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:25.389546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 682/1375] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:25.687737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69054fab-d696-486a-a1cb-47cb27bb453a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:25.689334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-69054fab-d696-486a-a1cb-47cb27bb453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:25.867972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69054fab-d696-486a-a1cb-47cb27bb453a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:25.923685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:25.990030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db671fb-b1ee-4f26-9fc8-87051cb00bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:25.990643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db671fb-b1ee-4f26-9fc8-87051cb00bcd service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:25.990915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db671fb-b1ee-4f26-9fc8-87051cb00bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:25.991180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db671fb-b1ee-4f26-9fc8-87051cb00bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:25.992790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9db671fb-b1ee-4f26-9fc8-87051cb00bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:26.087470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db671fb-b1ee-4f26-9fc8-87051cb00bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:26.089085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db671fb-b1ee-4f26-9fc8-87051cb00bcd service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:26.108798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 694/1376] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:28.451170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19e3cc-2e4b-4b86-92f6-17aefe998976 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:28.451744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19e3cc-2e4b-4b86-92f6-17aefe998976 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:28.451744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19e3cc-2e4b-4b86-92f6-17aefe998976 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:28.453521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 683/1377] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:28.460268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e017e33-27c9-4e4b-b912-7ccd0c3b9281 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:28.460268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e017e33-27c9-4e4b-b912-7ccd0c3b9281 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:28.460268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e017e33-27c9-4e4b-b912-7ccd0c3b9281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:28.549832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-9e017e33-27c9-4e4b-b912-7ccd0c3b9281 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:28.551040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9e017e33-27c9-4e4b-b912-7ccd0c3b9281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:28.559493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:28.659181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 695/1378] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:28.952183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19e3cc-2e4b-4b86-92f6-17aefe998976 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:28.954065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e19e3cc-2e4b-4b86-92f6-17aefe998976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:29.018676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:29.112281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19e3cc-2e4b-4b86-92f6-17aefe998976 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:29.237905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a4a44-4b8c-4221-a999-8f7f6d81f574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:29.238795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4a44-4b8c-4221-a999-8f7f6d81f574 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:29.239139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4a44-4b8c-4221-a999-8f7f6d81f574 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:29.239570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4a44-4b8c-4221-a999-8f7f6d81f574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:29.241240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc1a4a44-4b8c-4221-a999-8f7f6d81f574 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:29.261885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:29.341948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a4a44-4b8c-4221-a999-8f7f6d81f574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:29.343556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a4a44-4b8c-4221-a999-8f7f6d81f574 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:29.367520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 684/1379] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:31.626100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603173a6-f54b-4b73-8a64-f30ab30d5696 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:31.626536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603173a6-f54b-4b73-8a64-f30ab30d5696 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:31.626770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603173a6-f54b-4b73-8a64-f30ab30d5696 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:31.628567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 696/1380] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:31.633758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d68a7-ffb5-4a4c-a884-079288bf7e7d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:31.634206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d68a7-ffb5-4a4c-a884-079288bf7e7d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:31.634420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d68a7-ffb5-4a4c-a884-079288bf7e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:31.722202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-352d68a7-ffb5-4a4c-a884-079288bf7e7d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:31.724306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-352d68a7-ffb5-4a4c-a884-079288bf7e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:31.854556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:31.862063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 685/1381] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:32.159692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603173a6-f54b-4b73-8a64-f30ab30d5696 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:32.161835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-603173a6-f54b-4b73-8a64-f30ab30d5696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:32.348986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:32.366927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603173a6-f54b-4b73-8a64-f30ab30d5696 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:32.508721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48424e1-44b3-4749-a8a0-84738b59303a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:32.509374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48424e1-44b3-4749-a8a0-84738b59303a service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:32.509664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48424e1-44b3-4749-a8a0-84738b59303a service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:32.509964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48424e1-44b3-4749-a8a0-84738b59303a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:32.511599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d48424e1-44b3-4749-a8a0-84738b59303a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:32.605415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48424e1-44b3-4749-a8a0-84738b59303a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:32.607076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48424e1-44b3-4749-a8a0-84738b59303a service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:32.617148 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:32.626737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 697/1382] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:34.864263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b455d-34ba-4e1e-9600-58c413dc1d81 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:34.865090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b455d-34ba-4e1e-9600-58c413dc1d81 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:34.865090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b455d-34ba-4e1e-9600-58c413dc1d81 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:34.866667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 686/1383] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:34.873430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47cc98-1c8b-456e-9986-49ca4f6be870 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:34.874035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47cc98-1c8b-456e-9986-49ca4f6be870 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:34.874230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47cc98-1c8b-456e-9986-49ca4f6be870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:34.969469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-be47cc98-1c8b-456e-9986-49ca4f6be870 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:34.970779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-be47cc98-1c8b-456e-9986-49ca4f6be870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:35.074178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 698/1384] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:35.372075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b455d-34ba-4e1e-9600-58c413dc1d81 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:35.373632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a72b455d-34ba-4e1e-9600-58c413dc1d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:35.541562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b455d-34ba-4e1e-9600-58c413dc1d81 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:35.618599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:35.671512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3f793-4595-4eca-b493-936c0c1540ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:35.672314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3f793-4595-4eca-b493-936c0c1540ca service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:35.673597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3f793-4595-4eca-b493-936c0c1540ca service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:35.673927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3f793-4595-4eca-b493-936c0c1540ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:35.675646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b3c3f793-4595-4eca-b493-936c0c1540ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:35.799656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3f793-4595-4eca-b493-936c0c1540ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:35.801671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3f793-4595-4eca-b493-936c0c1540ca service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:35.826209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 687/1385] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:38.386965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f675-c90c-44b5-8b98-5fbc6ade0e03 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:38.387630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f675-c90c-44b5-8b98-5fbc6ade0e03 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:38.387630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f675-c90c-44b5-8b98-5fbc6ade0e03 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:38.389015 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 699/1386] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:50:38.394255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863c71-46f6-4c3d-9569-ebc41fe80887 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:38.394716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863c71-46f6-4c3d-9569-ebc41fe80887 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:38.394820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863c71-46f6-4c3d-9569-ebc41fe80887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:38.460705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a9863c71-46f6-4c3d-9569-ebc41fe80887 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:38.461955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a9863c71-46f6-4c3d-9569-ebc41fe80887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:38.495508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:38.563359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 688/1387] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:38.860732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319f675-c90c-44b5-8b98-5fbc6ade0e03 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:38.862358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c319f675-c90c-44b5-8b98-5fbc6ade0e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:38.945497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:39.020864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319f675-c90c-44b5-8b98-5fbc6ade0e03 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:39.147412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2f9ef-1b5e-4c47-a67d-dbf405b6d03d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:39.148220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2f9ef-1b5e-4c47-a67d-dbf405b6d03d service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:39.148347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2f9ef-1b5e-4c47-a67d-dbf405b6d03d service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:39.148717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2f9ef-1b5e-4c47-a67d-dbf405b6d03d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:39.150278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7bc2f9ef-1b5e-4c47-a67d-dbf405b6d03d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:39.199640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:39.254000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2f9ef-1b5e-4c47-a67d-dbf405b6d03d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:39.255590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2f9ef-1b5e-4c47-a67d-dbf405b6d03d service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:39.277026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 700/1388] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:39.601060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8264f0-59da-4483-98ef-ad34e1851209 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:39.601634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8264f0-59da-4483-98ef-ad34e1851209 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:39.601634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8264f0-59da-4483-98ef-ad34e1851209 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:39.603003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 689/1389] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:39.609008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062482d-1568-45c8-94c0-a8e93f0396d8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:39.609525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062482d-1568-45c8-94c0-a8e93f0396d8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:39.609654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062482d-1568-45c8-94c0-a8e93f0396d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:39.682227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a062482d-1568-45c8-94c0-a8e93f0396d8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:39.683485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a062482d-1568-45c8-94c0-a8e93f0396d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:39.782634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 701/1390] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:40.084506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eea172-e015-43c7-8762-cd984d83fb9d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:40.084862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eea172-e015-43c7-8762-cd984d83fb9d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:40.085250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eea172-e015-43c7-8762-cd984d83fb9d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:40.086672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 690/1391] 10.222.0.88 () {54 vars in 962 bytes} [Mon Sep 19 11:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:50:40.092023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc90f48-afba-41d0-a944-efff2aeefa59 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:40.092306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc90f48-afba-41d0-a944-efff2aeefa59 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:40.092646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc90f48-afba-41d0-a944-efff2aeefa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:40.108194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:40.175282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-1cc90f48-afba-41d0-a944-efff2aeefa59 None None] MFA Rules not processed for user `6a24bc4a0c4f480192d87a27a243acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:40.176478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1cc90f48-afba-41d0-a944-efff2aeefa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:40.278660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 702/1392] 10.222.0.88 () {58 vars in 1059 bytes} [Mon Sep 19 11:50:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:40.292700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eea172-e015-43c7-8762-cd984d83fb9d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:40.294730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-54eea172-e015-43c7-8762-cd984d83fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:40.438835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eea172-e015-43c7-8762-cd984d83fb9d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:40.505352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:40.520768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86818d-b318-42ea-a2ea-6b6d72b2a687 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a24bc4a0c4f480192d87a27a243acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:40.521378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86818d-b318-42ea-a2ea-6b6d72b2a687 service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:40.521660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86818d-b318-42ea-a2ea-6b6d72b2a687 service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:40.521962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86818d-b318-42ea-a2ea-6b6d72b2a687 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:40.523545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f86818d-b318-42ea-a2ea-6b6d72b2a687 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:40.648332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86818d-b318-42ea-a2ea-6b6d72b2a687 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:40.649799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86818d-b318-42ea-a2ea-6b6d72b2a687 service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:40.669085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 691/1393] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:41.443846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3696-fe9a-4ec5-a428-50a5516ce745 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:41.444361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3696-fe9a-4ec5-a428-50a5516ce745 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:41.444473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3696-fe9a-4ec5-a428-50a5516ce745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:41.520283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-768c3696-fe9a-4ec5-a428-50a5516ce745 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:41.521555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-768c3696-fe9a-4ec5-a428-50a5516ce745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:41.525283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf3740-4594-4aff-a66b-9a7e41be2d74 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:41.525568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf3740-4594-4aff-a66b-9a7e41be2d74 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:41.525878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf3740-4594-4aff-a66b-9a7e41be2d74 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:41.527282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 692/1394] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:41.534097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:41.534379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:41.534651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:41.586043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:41.618072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:41.619503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:41.624387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 703/1395] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:50:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:41.672709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:41.673361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4ef30-0d6b-4566-9d66-c3d749e0dfaf None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:41.673605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4ef30-0d6b-4566-9d66-c3d749e0dfaf None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:41.674074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4ef30-0d6b-4566-9d66-c3d749e0dfaf None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:41.675911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 704/1396] 10.222.0.88 () {54 vars in 943 bytes} [Mon Sep 19 11:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:50:41.683312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:41.683518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:41.684324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:41.751626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 693/1397] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:50:41.773971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] MFA Rules not processed for user `1924693bdf934b15b1a784abfefdea5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:41.775184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:41.887579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 705/1398] 10.222.0.88 () {58 vars in 1040 bytes} [Mon Sep 19 11:50:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:50:41.898932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:41.900150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:42.014249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b93c6e-539b-466a-8841-a883c7d2fd7e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:42.050283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:42.051495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:42.084409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:42.087455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:42.132730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40251524-32d5-4f2f-8993-f72aeb13e878 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:42.133338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40251524-32d5-4f2f-8993-f72aeb13e878 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:42.133614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40251524-32d5-4f2f-8993-f72aeb13e878 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:42.133908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40251524-32d5-4f2f-8993-f72aeb13e878 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:42.135574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40251524-32d5-4f2f-8993-f72aeb13e878 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:42.157920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d43a2-3a4c-4eb1-8b80-a51b686327fe None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:42.227229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40251524-32d5-4f2f-8993-f72aeb13e878 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:42.229123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40251524-32d5-4f2f-8993-f72aeb13e878 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:42.238383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 694/1399] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 11:50:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:50:42.302452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a75514-fc2b-4f43-ae69-e8ce50024636 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:42.303070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75514-fc2b-4f43-ae69-e8ce50024636 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:42.303349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75514-fc2b-4f43-ae69-e8ce50024636 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:42.303617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75514-fc2b-4f43-ae69-e8ce50024636 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:42.305277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d9a75514-fc2b-4f43-ae69-e8ce50024636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:42.363303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:42.425134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a75514-fc2b-4f43-ae69-e8ce50024636 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:42.426771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a75514-fc2b-4f43-ae69-e8ce50024636 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:42.448902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 706/1400] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:50:44.743664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ec74a-ae54-488a-8baa-9e27d6b51de9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:44.744234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ec74a-ae54-488a-8baa-9e27d6b51de9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:44.744234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ec74a-ae54-488a-8baa-9e27d6b51de9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:44.745859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 695/1401] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:44.752141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534818-e865-44b3-9cfb-1a5a2992670b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:44.752672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534818-e865-44b3-9cfb-1a5a2992670b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:44.752672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534818-e865-44b3-9cfb-1a5a2992670b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:44.835560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5d534818-e865-44b3-9cfb-1a5a2992670b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:44.836042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5d534818-e865-44b3-9cfb-1a5a2992670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:44.997932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 707/1402] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:45.331728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ec74a-ae54-488a-8baa-9e27d6b51de9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:45.333246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d60ec74a-ae54-488a-8baa-9e27d6b51de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:45.506658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:45.584685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ec74a-ae54-488a-8baa-9e27d6b51de9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:45.761409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c3db3-46a0-42ce-bdf7-68100ed63ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:45.761409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c3db3-46a0-42ce-bdf7-68100ed63ab7 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:45.761409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c3db3-46a0-42ce-bdf7-68100ed63ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:45.761409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c3db3-46a0-42ce-bdf7-68100ed63ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:45.762913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-968c3db3-46a0-42ce-bdf7-68100ed63ab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:45.863166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:45.901172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c3db3-46a0-42ce-bdf7-68100ed63ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:45.902885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c3db3-46a0-42ce-bdf7-68100ed63ab7 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:45.934157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 696/1403] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:48.207026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297efe1-76df-462b-9999-181235840739 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:48.208422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297efe1-76df-462b-9999-181235840739 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:48.208959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297efe1-76df-462b-9999-181235840739 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:48.219967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 708/1404] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:48.222006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f36d-62a3-4ebe-9743-86981310382f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:48.222817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f36d-62a3-4ebe-9743-86981310382f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:48.222961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f36d-62a3-4ebe-9743-86981310382f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:48.320110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b9f8f36d-62a3-4ebe-9743-86981310382f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:48.321749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9f8f36d-62a3-4ebe-9743-86981310382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:48.468737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 697/1405] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:48.760936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297efe1-76df-462b-9999-181235840739 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:48.762714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3297efe1-76df-462b-9999-181235840739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:48.784909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:48.929677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297efe1-76df-462b-9999-181235840739 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:49.031436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:49.060677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7be417-fa2f-4ebd-82e5-d72727b7ca24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:49.061386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7be417-fa2f-4ebd-82e5-d72727b7ca24 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:49.061725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7be417-fa2f-4ebd-82e5-d72727b7ca24 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:49.062078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7be417-fa2f-4ebd-82e5-d72727b7ca24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:49.063777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9e7be417-fa2f-4ebd-82e5-d72727b7ca24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:49.172592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7be417-fa2f-4ebd-82e5-d72727b7ca24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:49.174377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7be417-fa2f-4ebd-82e5-d72727b7ca24 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:49.196642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 709/1406] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:51.465619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44cc55-6f2f-44eb-8391-836601f8ff0e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:51.466226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44cc55-6f2f-44eb-8391-836601f8ff0e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:51.466226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44cc55-6f2f-44eb-8391-836601f8ff0e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:51.468026 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 698/1407] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:50:51.473511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a06ec3-b179-43b8-8e5e-6201caa49f02 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:51.474031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a06ec3-b179-43b8-8e5e-6201caa49f02 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:51.474136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a06ec3-b179-43b8-8e5e-6201caa49f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:51.554081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-31a06ec3-b179-43b8-8e5e-6201caa49f02 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:50:51.555336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-31a06ec3-b179-43b8-8e5e-6201caa49f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:51.623916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:51.681180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 710/1408] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:50:51.987985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44cc55-6f2f-44eb-8391-836601f8ff0e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:51.989687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c44cc55-6f2f-44eb-8391-836601f8ff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:52.051142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:52.174936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44cc55-6f2f-44eb-8391-836601f8ff0e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:52.309875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2366530-b96a-4b84-b640-05fdde21c20a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:52.310495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2366530-b96a-4b84-b640-05fdde21c20a service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:52.310798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2366530-b96a-4b84-b640-05fdde21c20a service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:52.311050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2366530-b96a-4b84-b640-05fdde21c20a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:52.313752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f2366530-b96a-4b84-b640-05fdde21c20a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:52.315886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:52.427793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2366530-b96a-4b84-b640-05fdde21c20a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:52.429684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2366530-b96a-4b84-b640-05fdde21c20a service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:52.449106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 699/1409] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:01.679540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=19 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:02.446525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=44 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:05.181438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa92161-68a5-43af-96fd-8f87d44bdd28 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:05.182009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa92161-68a5-43af-96fd-8f87d44bdd28 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:05.182009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa92161-68a5-43af-96fd-8f87d44bdd28 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:05.183351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 711/1410] 10.222.0.88 () {54 vars in 918 bytes} [Mon Sep 19 11:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:05.189790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb194dd-47d1-49be-934a-837d97c29d26 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:05.190361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb194dd-47d1-49be-934a-837d97c29d26 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:05.190858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb194dd-47d1-49be-934a-837d97c29d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:05.272143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-afb194dd-47d1-49be-934a-837d97c29d26 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:05.273354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-afb194dd-47d1-49be-934a-837d97c29d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:05.383076 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 700/1411] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Sep 19 11:51:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:05.397812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97dfb3-508a-4b8b-8bb9-04f766fa9211 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:05.398121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97dfb3-508a-4b8b-8bb9-04f766fa9211 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:05.398419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97dfb3-508a-4b8b-8bb9-04f766fa9211 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:05.399741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 712/1412] 10.222.0.88 () {54 vars in 918 bytes} [Mon Sep 19 11:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:05.405372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dd7d3-13c6-4af7-9e02-4bda1b0654f2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:05.405652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dd7d3-13c6-4af7-9e02-4bda1b0654f2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:05.405952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dd7d3-13c6-4af7-9e02-4bda1b0654f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:05.478362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:05.481157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2e4dd7d3-13c6-4af7-9e02-4bda1b0654f2 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:05.482985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2e4dd7d3-13c6-4af7-9e02-4bda1b0654f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:05.597733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 701/1413] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Sep 19 11:51:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:08.219489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c71770-dfbd-49e9-9467-e70ba9603344 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:08.220052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c71770-dfbd-49e9-9467-e70ba9603344 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:08.220052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c71770-dfbd-49e9-9467-e70ba9603344 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:08.221365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:08.226964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0caea1-4a15-46b9-9a69-d3c46f037c35 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:08.227443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0caea1-4a15-46b9-9a69-d3c46f037c35 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:08.227540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0caea1-4a15-46b9-9a69-d3c46f037c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:08.293406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bd0caea1-4a15-46b9-9a69-d3c46f037c35 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:08.295167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd0caea1-4a15-46b9-9a69-d3c46f037c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:08.351240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:08.393884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:08.686938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c71770-dfbd-49e9-9467-e70ba9603344 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:08.688464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-02c71770-dfbd-49e9-9467-e70ba9603344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:08.834826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c71770-dfbd-49e9-9467-e70ba9603344 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:08.915139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:09.073997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7335c-1383-4bf9-a894-1771077c56e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:09.074597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7335c-1383-4bf9-a894-1771077c56e2 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:09.074887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7335c-1383-4bf9-a894-1771077c56e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:09.075134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7335c-1383-4bf9-a894-1771077c56e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:09.077338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1fd7335c-1383-4bf9-a894-1771077c56e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:09.174484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7335c-1383-4bf9-a894-1771077c56e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:09.176660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7335c-1383-4bf9-a894-1771077c56e2 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:09.196680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 714/1416] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:10.361486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfee61a-390d-4fae-b6d1-09803439d8c0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:10.362109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfee61a-390d-4fae-b6d1-09803439d8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:10.362233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfee61a-390d-4fae-b6d1-09803439d8c0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:10.363590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:10.370181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b315f7-ddc3-4dcf-8abb-09d95d0bb0bf None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:10.370631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b315f7-ddc3-4dcf-8abb-09d95d0bb0bf None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:10.370750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b315f7-ddc3-4dcf-8abb-09d95d0bb0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:10.446442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-88b315f7-ddc3-4dcf-8abb-09d95d0bb0bf None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:10.447675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-88b315f7-ddc3-4dcf-8abb-09d95d0bb0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:10.474603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:10.544081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:10.695386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfee61a-390d-4fae-b6d1-09803439d8c0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:10.696838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ccfee61a-390d-4fae-b6d1-09803439d8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:10.841102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfee61a-390d-4fae-b6d1-09803439d8c0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:10.880978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:10.954914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fec77-e3d5-4605-b41b-44a0dbbdf445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:10.955430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fec77-e3d5-4605-b41b-44a0dbbdf445 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:10.955708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fec77-e3d5-4605-b41b-44a0dbbdf445 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:10.955978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fec77-e3d5-4605-b41b-44a0dbbdf445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:10.957586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bb1fec77-e3d5-4605-b41b-44a0dbbdf445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:11.046935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fec77-e3d5-4605-b41b-44a0dbbdf445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:11.048530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fec77-e3d5-4605-b41b-44a0dbbdf445 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:11.066674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 704/1419] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:12.243896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4497b-eda3-4b0c-b092-18edbca29266 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:12.244464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4497b-eda3-4b0c-b092-18edbca29266 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:12.244464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4497b-eda3-4b0c-b092-18edbca29266 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:12.245667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:12.252754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbc92-3e17-499f-985a-b322d6b58e17 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:12.253253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbc92-3e17-499f-985a-b322d6b58e17 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:12.253360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbc92-3e17-499f-985a-b322d6b58e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:12.308984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:12.330924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-770bbc92-3e17-499f-985a-b322d6b58e17 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:12.332137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-770bbc92-3e17-499f-985a-b322d6b58e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:12.429779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:12.580652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4497b-eda3-4b0c-b092-18edbca29266 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:12.582210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-bed4497b-eda3-4b0c-b092-18edbca29266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:12.728932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4497b-eda3-4b0c-b092-18edbca29266 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:12.740691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:12.841447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1b220-e81e-4154-b2d0-f982c579f7d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:12.842526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1b220-e81e-4154-b2d0-f982c579f7d5 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:12.842806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1b220-e81e-4154-b2d0-f982c579f7d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:12.843075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1b220-e81e-4154-b2d0-f982c579f7d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:12.844677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-23b1b220-e81e-4154-b2d0-f982c579f7d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:12.931893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1b220-e81e-4154-b2d0-f982c579f7d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:12.933473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1b220-e81e-4154-b2d0-f982c579f7d5 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:12.952464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 717/1422] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:14.095357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34a0e2-1eda-41bb-a07d-a620215ac627 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:14.095933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34a0e2-1eda-41bb-a07d-a620215ac627 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:14.095933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34a0e2-1eda-41bb-a07d-a620215ac627 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:14.097247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:14.103739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d74fbe-453e-4bd9-8b94-7bc2c7a16e30 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:14.104198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d74fbe-453e-4bd9-8b94-7bc2c7a16e30 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:14.104311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d74fbe-453e-4bd9-8b94-7bc2c7a16e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:14.124629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:14.182418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-22d74fbe-453e-4bd9-8b94-7bc2c7a16e30 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:14.183626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-22d74fbe-453e-4bd9-8b94-7bc2c7a16e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:14.278803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:14.427948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34a0e2-1eda-41bb-a07d-a620215ac627 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:14.429488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc34a0e2-1eda-41bb-a07d-a620215ac627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:14.540787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:14.576358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34a0e2-1eda-41bb-a07d-a620215ac627 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:14.689381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3a479-ac09-4712-b6ca-84bd5273dc34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:14.690013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3a479-ac09-4712-b6ca-84bd5273dc34 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:14.690294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3a479-ac09-4712-b6ca-84bd5273dc34 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:14.690567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3a479-ac09-4712-b6ca-84bd5273dc34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:14.692163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e6d3a479-ac09-4712-b6ca-84bd5273dc34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:14.767761 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:14.781640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3a479-ac09-4712-b6ca-84bd5273dc34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:14.783246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3a479-ac09-4712-b6ca-84bd5273dc34 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:14.801069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 707/1425] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:15.932687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429e1e-181b-440c-a4bd-5f9c53b04db7 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:15.933283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429e1e-181b-440c-a4bd-5f9c53b04db7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:15.933283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429e1e-181b-440c-a4bd-5f9c53b04db7 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:15.934635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:15.941265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ead6-e4d6-423a-9b65-67ec5f412905 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:15.941754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ead6-e4d6-423a-9b65-67ec5f412905 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:15.941889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ead6-e4d6-423a-9b65-67ec5f412905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:16.019240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5c05ead6-e4d6-423a-9b65-67ec5f412905 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:16.020503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c05ead6-e4d6-423a-9b65-67ec5f412905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:16.126557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:16.288799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f429e1e-181b-440c-a4bd-5f9c53b04db7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:16.290393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5f429e1e-181b-440c-a4bd-5f9c53b04db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:16.383114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:16.452087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f429e1e-181b-440c-a4bd-5f9c53b04db7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:16.567464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d15d9-a330-4e21-a17a-a777d861e4fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:16.568063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d15d9-a330-4e21-a17a-a777d861e4fe service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:16.568349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d15d9-a330-4e21-a17a-a777d861e4fe service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:16.568630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d15d9-a330-4e21-a17a-a777d861e4fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:16.570324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-652d15d9-a330-4e21-a17a-a777d861e4fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:16.618748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:16.659756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d15d9-a330-4e21-a17a-a777d861e4fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:16.661341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d15d9-a330-4e21-a17a-a777d861e4fe service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:16.679640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 720/1428] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:17.752489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27df2-0f17-47aa-be17-03c459a42ed3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:17.753061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27df2-0f17-47aa-be17-03c459a42ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:17.753061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27df2-0f17-47aa-be17-03c459a42ed3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:17.754509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:17.763680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c5fb-fc33-4d21-972f-3027f8a0f090 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:17.764225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c5fb-fc33-4d21-972f-3027f8a0f090 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:17.764550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c5fb-fc33-4d21-972f-3027f8a0f090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:17.844677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0542c5fb-fc33-4d21-972f-3027f8a0f090 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:17.845807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0542c5fb-fc33-4d21-972f-3027f8a0f090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:17.939758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:18.106249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f27df2-0f17-47aa-be17-03c459a42ed3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:18.108165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c3f27df2-0f17-47aa-be17-03c459a42ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:18.160564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:18.257506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f27df2-0f17-47aa-be17-03c459a42ed3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:18.368942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb8d93-0980-4aff-9fba-7f80f21dbaf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:18.369454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb8d93-0980-4aff-9fba-7f80f21dbaf6 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:18.369729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb8d93-0980-4aff-9fba-7f80f21dbaf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:18.370031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb8d93-0980-4aff-9fba-7f80f21dbaf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:18.371646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dccb8d93-0980-4aff-9fba-7f80f21dbaf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:18.378640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:18.465287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb8d93-0980-4aff-9fba-7f80f21dbaf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:18.467399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb8d93-0980-4aff-9fba-7f80f21dbaf6 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:18.485611 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 710/1431] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:19.557689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9208d-3e3e-4d8a-bb1a-a36d0251888a None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:19.558283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9208d-3e3e-4d8a-bb1a-a36d0251888a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:19.558283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9208d-3e3e-4d8a-bb1a-a36d0251888a None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:19.559641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:19.566893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31b1f-ec38-409a-b766-2b89683f8d5b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:19.567365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31b1f-ec38-409a-b766-2b89683f8d5b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:19.567483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31b1f-ec38-409a-b766-2b89683f8d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:19.642686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-92c31b1f-ec38-409a-b766-2b89683f8d5b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:19.644028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-92c31b1f-ec38-409a-b766-2b89683f8d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:19.712723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:19.741921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:19.892727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9208d-3e3e-4d8a-bb1a-a36d0251888a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:19.894213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a3c9208d-3e3e-4d8a-bb1a-a36d0251888a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:19.911368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:20.042865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9208d-3e3e-4d8a-bb1a-a36d0251888a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:20.137643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:20.167859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1abe4-9ae1-4aa7-8639-b52299944570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:20.168468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1abe4-9ae1-4aa7-8639-b52299944570 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:20.168748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1abe4-9ae1-4aa7-8639-b52299944570 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:20.169022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1abe4-9ae1-4aa7-8639-b52299944570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:20.171200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2ca1abe4-9ae1-4aa7-8639-b52299944570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:20.264675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1abe4-9ae1-4aa7-8639-b52299944570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:20.266556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1abe4-9ae1-4aa7-8639-b52299944570 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:20.285495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 723/1434] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:21.355097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb241c-cb34-47e5-901e-0404ae8676f6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:21.355654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb241c-cb34-47e5-901e-0404ae8676f6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:21.355654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb241c-cb34-47e5-901e-0404ae8676f6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:21.356991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:21.364326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4b388-3b38-4be1-915d-99dd26c56681 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:21.364880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4b388-3b38-4be1-915d-99dd26c56681 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:21.364880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4b388-3b38-4be1-915d-99dd26c56681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:21.443098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-86a4b388-3b38-4be1-915d-99dd26c56681 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:21.444223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-86a4b388-3b38-4be1-915d-99dd26c56681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:21.485738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:21.541783 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:21.691660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb241c-cb34-47e5-901e-0404ae8676f6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:21.693351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ebdb241c-cb34-47e5-901e-0404ae8676f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:21.855822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb241c-cb34-47e5-901e-0404ae8676f6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:21.910137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:21.969210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd66daa-19eb-48fe-bf5a-40b91650ad72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:21.969727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd66daa-19eb-48fe-bf5a-40b91650ad72 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:21.970037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd66daa-19eb-48fe-bf5a-40b91650ad72 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:21.970308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd66daa-19eb-48fe-bf5a-40b91650ad72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:21.971898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fcd66daa-19eb-48fe-bf5a-40b91650ad72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:22.064901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd66daa-19eb-48fe-bf5a-40b91650ad72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:22.066511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd66daa-19eb-48fe-bf5a-40b91650ad72 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:22.085602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 713/1437] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:23.151299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc9a4-6b08-4137-ada7-be321f623404 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:23.151795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc9a4-6b08-4137-ada7-be321f623404 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:23.151893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc9a4-6b08-4137-ada7-be321f623404 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:23.153173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:23.160546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b674-b7b8-4471-89cf-dc8a637a0c51 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:23.161039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b674-b7b8-4471-89cf-dc8a637a0c51 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:23.161155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b674-b7b8-4471-89cf-dc8a637a0c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:23.236490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e014b674-b7b8-4471-89cf-dc8a637a0c51 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:23.237686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e014b674-b7b8-4471-89cf-dc8a637a0c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:23.239724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:23.333914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:23.484375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dc9a4-6b08-4137-ada7-be321f623404 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:23.485860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ea7dc9a4-6b08-4137-ada7-be321f623404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:23.631404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dc9a4-6b08-4137-ada7-be321f623404 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:23.658638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:23.743572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ef31e-4e3a-4466-b8b0-7313d30bbbfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:23.744092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ef31e-4e3a-4466-b8b0-7313d30bbbfb service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:23.744370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ef31e-4e3a-4466-b8b0-7313d30bbbfb service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:23.744642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ef31e-4e3a-4466-b8b0-7313d30bbbfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:23.746276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f99ef31e-4e3a-4466-b8b0-7313d30bbbfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:23.837215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ef31e-4e3a-4466-b8b0-7313d30bbbfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:23.838884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ef31e-4e3a-4466-b8b0-7313d30bbbfb service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:23.858090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 726/1440] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:25.960624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8193c5-0534-4c4b-918f-587499d8d360 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:25.961176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8193c5-0534-4c4b-918f-587499d8d360 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:25.961480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8193c5-0534-4c4b-918f-587499d8d360 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:25.963167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:25.968542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b820c-eefc-474c-855b-6d94169c05bd None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:25.969012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b820c-eefc-474c-855b-6d94169c05bd None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:25.969126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b820c-eefc-474c-855b-6d94169c05bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:26.011247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:26.037869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-9e7b820c-eefc-474c-855b-6d94169c05bd None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:26.039100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9e7b820c-eefc-474c-855b-6d94169c05bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:26.140280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:26.457247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8193c5-0534-4c4b-918f-587499d8d360 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:26.458829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9e8193c5-0534-4c4b-918f-587499d8d360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:26.570632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8193c5-0534-4c4b-918f-587499d8d360 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:26.605507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:26.648223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801a6f8-7367-4cb9-90f5-556f3f1e844f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:26.648826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a6f8-7367-4cb9-90f5-556f3f1e844f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:26.649103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a6f8-7367-4cb9-90f5-556f3f1e844f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:26.649381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a6f8-7367-4cb9-90f5-556f3f1e844f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:26.651078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5801a6f8-7367-4cb9-90f5-556f3f1e844f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:26.760849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801a6f8-7367-4cb9-90f5-556f3f1e844f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:26.762624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801a6f8-7367-4cb9-90f5-556f3f1e844f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:26.781164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 716/1443] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:29.220587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3026f-ac17-45bb-8175-be3920f50a42 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:29.221163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3026f-ac17-45bb-8175-be3920f50a42 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:29.221620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3026f-ac17-45bb-8175-be3920f50a42 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:29.223190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:29.228927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997d4e8-70ad-4bb6-b8a0-45987bd9561d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:29.229432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997d4e8-70ad-4bb6-b8a0-45987bd9561d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:29.229596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997d4e8-70ad-4bb6-b8a0-45987bd9561d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:29.301729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b997d4e8-70ad-4bb6-b8a0-45987bd9561d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:29.302995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b997d4e8-70ad-4bb6-b8a0-45987bd9561d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:29.315688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:29.401106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:29.693008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3026f-ac17-45bb-8175-be3920f50a42 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:29.694697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-96b3026f-ac17-45bb-8175-be3920f50a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:29.798375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3026f-ac17-45bb-8175-be3920f50a42 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:29.810294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:29.872823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14a725-607a-48dc-b2c4-3fdfd0e00372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:29.873946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14a725-607a-48dc-b2c4-3fdfd0e00372 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:29.874225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14a725-607a-48dc-b2c4-3fdfd0e00372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:29.874493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14a725-607a-48dc-b2c4-3fdfd0e00372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:29.876085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9e14a725-607a-48dc-b2c4-3fdfd0e00372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:29.990001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14a725-607a-48dc-b2c4-3fdfd0e00372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:29.991307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14a725-607a-48dc-b2c4-3fdfd0e00372 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:30.009568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 729/1446] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:32.629433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b80eb-6527-4aa2-9135-67c1cfb2ea5d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:32.630038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b80eb-6527-4aa2-9135-67c1cfb2ea5d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:32.630038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b80eb-6527-4aa2-9135-67c1cfb2ea5d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:32.631420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:32.636771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b84a-a2ad-4886-a4ad-65b9178f4d59 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:32.637160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b84a-a2ad-4886-a4ad-65b9178f4d59 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:32.637332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b84a-a2ad-4886-a4ad-65b9178f4d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:32.685049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:32.704735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4801b84a-a2ad-4886-a4ad-65b9178f4d59 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:32.706223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4801b84a-a2ad-4886-a4ad-65b9178f4d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:32.801592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:33.092346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b80eb-6527-4aa2-9135-67c1cfb2ea5d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:33.093932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b93b80eb-6527-4aa2-9135-67c1cfb2ea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:33.200452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b80eb-6527-4aa2-9135-67c1cfb2ea5d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:33.242197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:33.281310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee7bdf-be47-4117-b223-a7013f41395b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:33.281940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee7bdf-be47-4117-b223-a7013f41395b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:33.282220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee7bdf-be47-4117-b223-a7013f41395b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:33.282486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee7bdf-be47-4117-b223-a7013f41395b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:33.284104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-93ee7bdf-be47-4117-b223-a7013f41395b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:33.399553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee7bdf-be47-4117-b223-a7013f41395b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:33.400938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee7bdf-be47-4117-b223-a7013f41395b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:33.418660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 719/1449] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:35.743750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d97a4-25ed-48ea-9d8b-4f244ea34e9c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:35.744344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d97a4-25ed-48ea-9d8b-4f244ea34e9c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:35.744344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d97a4-25ed-48ea-9d8b-4f244ea34e9c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:35.746020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:35.751411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63db01-5fd1-418e-8814-af64e59154c9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:35.751848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63db01-5fd1-418e-8814-af64e59154c9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:35.751987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63db01-5fd1-418e-8814-af64e59154c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:35.819476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5d63db01-5fd1-418e-8814-af64e59154c9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:35.820694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d63db01-5fd1-418e-8814-af64e59154c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:35.842109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:35.921134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:36.219366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d97a4-25ed-48ea-9d8b-4f244ea34e9c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:36.221130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e47d97a4-25ed-48ea-9d8b-4f244ea34e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:36.329851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d97a4-25ed-48ea-9d8b-4f244ea34e9c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:36.352294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:36.410952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71998f16-4436-484c-9a99-78f10771e15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:36.411675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71998f16-4436-484c-9a99-78f10771e15e service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:36.411960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71998f16-4436-484c-9a99-78f10771e15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:36.412263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71998f16-4436-484c-9a99-78f10771e15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:36.414128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-71998f16-4436-484c-9a99-78f10771e15e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:36.536223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71998f16-4436-484c-9a99-78f10771e15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:36.537734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71998f16-4436-484c-9a99-78f10771e15e service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:36.557079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 732/1452] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:38.779114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800edb3-0aa9-46f3-829e-8ff43c220db0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:38.779687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800edb3-0aa9-46f3-829e-8ff43c220db0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:38.779687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800edb3-0aa9-46f3-829e-8ff43c220db0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:38.781030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:38.786609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fffffc-6a49-4900-be04-6d146d80908e None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:38.787004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fffffc-6a49-4900-be04-6d146d80908e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:38.787184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fffffc-6a49-4900-be04-6d146d80908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:38.850618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:38.854556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a7fffffc-6a49-4900-be04-6d146d80908e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:38.855766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a7fffffc-6a49-4900-be04-6d146d80908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:38.952059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:39.238724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800edb3-0aa9-46f3-829e-8ff43c220db0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:39.240269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4800edb3-0aa9-46f3-829e-8ff43c220db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:39.343235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800edb3-0aa9-46f3-829e-8ff43c220db0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:39.392443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:39.416938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469f726-9eed-4c29-a24f-951ce3245c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:39.417474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469f726-9eed-4c29-a24f-951ce3245c45 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:39.417751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469f726-9eed-4c29-a24f-951ce3245c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:39.418055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469f726-9eed-4c29-a24f-951ce3245c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:39.419633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1469f726-9eed-4c29-a24f-951ce3245c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:39.527170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469f726-9eed-4c29-a24f-951ce3245c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:39.528589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469f726-9eed-4c29-a24f-951ce3245c45 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:39.546913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 722/1455] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:41.785121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9dad-7a0e-418d-9026-11dcf981a8ee None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:41.785658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9dad-7a0e-418d-9026-11dcf981a8ee None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:41.785658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9dad-7a0e-418d-9026-11dcf981a8ee None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:41.787025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:41.792321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003073f-0f2a-4713-a0b9-ee75f46df5c1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:41.792754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003073f-0f2a-4713-a0b9-ee75f46df5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:41.792854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003073f-0f2a-4713-a0b9-ee75f46df5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:41.863849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7003073f-0f2a-4713-a0b9-ee75f46df5c1 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:41.865048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7003073f-0f2a-4713-a0b9-ee75f46df5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:41.893473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:41.964551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:42.260240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9dad-7a0e-418d-9026-11dcf981a8ee None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:42.261812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9c1c9dad-7a0e-418d-9026-11dcf981a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:42.368068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9dad-7a0e-418d-9026-11dcf981a8ee None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:42.396874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:42.442717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3776f06-dca6-4e2e-a587-829369590fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:42.443319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3776f06-dca6-4e2e-a587-829369590fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:42.443597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3776f06-dca6-4e2e-a587-829369590fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:42.443869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3776f06-dca6-4e2e-a587-829369590fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:42.445470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f3776f06-dca6-4e2e-a587-829369590fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:42.554189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3776f06-dca6-4e2e-a587-829369590fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:42.555613 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3776f06-dca6-4e2e-a587-829369590fa0 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:42.573225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 735/1458] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:44.830279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b36fd-f0aa-4e4b-897a-05031e4098ab None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:44.830826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b36fd-f0aa-4e4b-897a-05031e4098ab None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:44.830826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b36fd-f0aa-4e4b-897a-05031e4098ab None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:44.832431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:44.838143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc313b-1552-4d87-9bf5-362a9996f6e1 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:44.838635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc313b-1552-4d87-9bf5-362a9996f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:44.838744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc313b-1552-4d87-9bf5-362a9996f6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:44.906521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-14bc313b-1552-4d87-9bf5-362a9996f6e1 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:44.907742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-14bc313b-1552-4d87-9bf5-362a9996f6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:44.916056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:45.009351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:51:45.300404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b36fd-f0aa-4e4b-897a-05031e4098ab None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:45.301958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-465b36fd-f0aa-4e4b-897a-05031e4098ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:45.428780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b36fd-f0aa-4e4b-897a-05031e4098ab None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:45.483707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:45.502766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c153f-7173-47a6-9d03-4837a9f3ec92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:45.503372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c153f-7173-47a6-9d03-4837a9f3ec92 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:45.503644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c153f-7173-47a6-9d03-4837a9f3ec92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:45.503912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c153f-7173-47a6-9d03-4837a9f3ec92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:45.505514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-999c153f-7173-47a6-9d03-4837a9f3ec92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:45.612496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c153f-7173-47a6-9d03-4837a9f3ec92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:45.613930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c153f-7173-47a6-9d03-4837a9f3ec92 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:45.631829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 725/1461] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:47.860909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d27bd9-74ee-4eea-9e74-b21096aea012 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:47.861487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d27bd9-74ee-4eea-9e74-b21096aea012 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:47.861487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d27bd9-74ee-4eea-9e74-b21096aea012 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:47.862854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:47.868509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a85fc-6203-463f-be65-7464b73acbbf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:47.868946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a85fc-6203-463f-be65-7464b73acbbf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:47.869046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a85fc-6203-463f-be65-7464b73acbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:47.934199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ae8a85fc-6203-463f-be65-7464b73acbbf None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:47.935402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ae8a85fc-6203-463f-be65-7464b73acbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:47.973336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:48.031036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:48.315277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d27bd9-74ee-4eea-9e74-b21096aea012 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:48.316811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-04d27bd9-74ee-4eea-9e74-b21096aea012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:48.425255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d27bd9-74ee-4eea-9e74-b21096aea012 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:48.469412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:48.505359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa683-7f39-48e2-a2d9-f758b15fb925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:48.505975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa683-7f39-48e2-a2d9-f758b15fb925 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:48.506249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa683-7f39-48e2-a2d9-f758b15fb925 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:48.506511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa683-7f39-48e2-a2d9-f758b15fb925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:48.508088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-018aa683-7f39-48e2-a2d9-f758b15fb925 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:48.615283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aa683-7f39-48e2-a2d9-f758b15fb925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:48.616696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aa683-7f39-48e2-a2d9-f758b15fb925 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:48.634266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 738/1464] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:50.859128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c634440-632d-4521-b0b8-117da803e0b5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:50.859672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c634440-632d-4521-b0b8-117da803e0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:50.859672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c634440-632d-4521-b0b8-117da803e0b5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:50.861111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:50.866782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dae56-4a09-4d52-a7bd-304d7f5c4475 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:50.867210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dae56-4a09-4d52-a7bd-304d7f5c4475 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:50.867308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dae56-4a09-4d52-a7bd-304d7f5c4475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:50.932135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3f1dae56-4a09-4d52-a7bd-304d7f5c4475 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:50.933332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3f1dae56-4a09-4d52-a7bd-304d7f5c4475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:50.951338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:51.028078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:51.316826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c634440-632d-4521-b0b8-117da803e0b5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:51.318426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9c634440-632d-4521-b0b8-117da803e0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:51.425584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c634440-632d-4521-b0b8-117da803e0b5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:51.493504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:51.500757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec09505-efa3-40fc-8e7a-ad8ab53596ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:51.501663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09505-efa3-40fc-8e7a-ad8ab53596ba service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:51.501974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09505-efa3-40fc-8e7a-ad8ab53596ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:51.502447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09505-efa3-40fc-8e7a-ad8ab53596ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:51.504950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cec09505-efa3-40fc-8e7a-ad8ab53596ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:51.615687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec09505-efa3-40fc-8e7a-ad8ab53596ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:51.617119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec09505-efa3-40fc-8e7a-ad8ab53596ba service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:51.635181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 728/1467] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:53.874031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1419d-153d-4be9-b665-660ff3840c8c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:53.874677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1419d-153d-4be9-b665-660ff3840c8c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:53.874677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1419d-153d-4be9-b665-660ff3840c8c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:53.875940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:51:53.881469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bda341-ac08-4461-b8c8-febaae5e0b86 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:53.881979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bda341-ac08-4461-b8c8-febaae5e0b86 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:53.882232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bda341-ac08-4461-b8c8-febaae5e0b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:53.950250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-72bda341-ac08-4461-b8c8-febaae5e0b86 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:51:53.951541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-72bda341-ac08-4461-b8c8-febaae5e0b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:54.000419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:54.049787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:51:54.331989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1419d-153d-4be9-b665-660ff3840c8c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:54.333526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-56f1419d-153d-4be9-b665-660ff3840c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:54.440677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1419d-153d-4be9-b665-660ff3840c8c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:54.492255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:54.518396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b658c-0d95-4a3b-a1aa-7a67cb3dae84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:54.519016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b658c-0d95-4a3b-a1aa-7a67cb3dae84 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:54.519288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b658c-0d95-4a3b-a1aa-7a67cb3dae84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:54.519551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b658c-0d95-4a3b-a1aa-7a67cb3dae84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:54.521139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-307b658c-0d95-4a3b-a1aa-7a67cb3dae84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:54.633539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b658c-0d95-4a3b-a1aa-7a67cb3dae84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:54.635210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b658c-0d95-4a3b-a1aa-7a67cb3dae84 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:54.656027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 741/1470] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:04.048124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=22 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:04.652786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=66 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:14.785166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9889fe-9dd3-4da2-ad43-bc1d8f12c2c1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:14.785736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9889fe-9dd3-4da2-ad43-bc1d8f12c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:14.785736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9889fe-9dd3-4da2-ad43-bc1d8f12c2c1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:14.787093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 730/1471] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:52:14.792354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6839-7243-4c29-97d9-e79803d75822 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:14.792770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6839-7243-4c29-97d9-e79803d75822 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:14.792900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6839-7243-4c29-97d9-e79803d75822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:14.862186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8bfb6839-7243-4c29-97d9-e79803d75822 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:14.863804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8bfb6839-7243-4c29-97d9-e79803d75822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:14.959093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 742/1472] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:52:15.256336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a743d-d5ea-4d4c-9403-b393abb299ce None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:15.256622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a743d-d5ea-4d4c-9403-b393abb299ce None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:15.256896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a743d-d5ea-4d4c-9403-b393abb299ce None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:15.258270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 731/1473] 10.222.0.88 () {54 vars in 962 bytes} [Mon Sep 19 11:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:52:15.263320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656238b-3480-4bd0-807f-18b66cb1636f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:15.263605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656238b-3480-4bd0-807f-18b66cb1636f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:15.264456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656238b-3480-4bd0-807f-18b66cb1636f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:15.342763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2656238b-3480-4bd0-807f-18b66cb1636f None None] MFA Rules not processed for user `6a24bc4a0c4f480192d87a27a243acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:15.343991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2656238b-3480-4bd0-807f-18b66cb1636f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:15.369066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:15.425864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 743/1474] 10.222.0.88 () {58 vars in 1059 bytes} [Mon Sep 19 11:52:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:52:15.433691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a743d-d5ea-4d4c-9403-b393abb299ce None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:15.435258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f07a743d-d5ea-4d4c-9403-b393abb299ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:15.539225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a743d-d5ea-4d4c-9403-b393abb299ce None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:15.614569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f3cda-0dd8-4065-a360-001dc6920e0b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a24bc4a0c4f480192d87a27a243acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:15.615168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3cda-0dd8-4065-a360-001dc6920e0b service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:15.615446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3cda-0dd8-4065-a360-001dc6920e0b service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:15.615715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3cda-0dd8-4065-a360-001dc6920e0b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:15.617307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-076f3cda-0dd8-4065-a360-001dc6920e0b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:15.680143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:15.742912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f3cda-0dd8-4065-a360-001dc6920e0b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:15.744352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f3cda-0dd8-4065-a360-001dc6920e0b service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:15.763805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 732/1475] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:18.054642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167619f-d9bf-4d1a-a1b4-513d84596cce None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:18.055189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167619f-d9bf-4d1a-a1b4-513d84596cce None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:18.055302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167619f-d9bf-4d1a-a1b4-513d84596cce None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:18.056581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 744/1476] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:52:18.061814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a1f8d-31cd-469a-b4be-41c1b6ea8873 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:18.062264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a1f8d-31cd-469a-b4be-41c1b6ea8873 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:18.062456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a1f8d-31cd-469a-b4be-41c1b6ea8873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:18.130588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7d4a1f8d-31cd-469a-b4be-41c1b6ea8873 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:18.131803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7d4a1f8d-31cd-469a-b4be-41c1b6ea8873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:18.235555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 733/1477] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:52:18.514753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167619f-d9bf-4d1a-a1b4-513d84596cce None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:18.516260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1167619f-d9bf-4d1a-a1b4-513d84596cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:18.641905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167619f-d9bf-4d1a-a1b4-513d84596cce None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:18.691424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:18.735720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63930e0f-6a86-4b23-a1c2-5dd9b79b3fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:18.736323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930e0f-6a86-4b23-a1c2-5dd9b79b3fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:18.736595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930e0f-6a86-4b23-a1c2-5dd9b79b3fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:18.736858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930e0f-6a86-4b23-a1c2-5dd9b79b3fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:18.737491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63930e0f-6a86-4b23-a1c2-5dd9b79b3fa6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:18.739244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-63930e0f-6a86-4b23-a1c2-5dd9b79b3fa6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:52:18.739244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:52:18.739244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:52:18.740826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63930e0f-6a86-4b23-a1c2-5dd9b79b3fa6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:18.760085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 745/1478] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Sep 19 11:52:18 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:52:20.955033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f24ad6-db16-4f84-9fcd-61886d6ba291 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:20.955629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f24ad6-db16-4f84-9fcd-61886d6ba291 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:20.955629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f24ad6-db16-4f84-9fcd-61886d6ba291 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:20.956920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 734/1479] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:52:20.962646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6f746-a589-4e0b-b506-0a1a43d3d997 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:20.963089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6f746-a589-4e0b-b506-0a1a43d3d997 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:20.963264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6f746-a589-4e0b-b506-0a1a43d3d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:21.029285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-47f6f746-a589-4e0b-b506-0a1a43d3d997 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:21.030556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-47f6f746-a589-4e0b-b506-0a1a43d3d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:21.127601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 746/1480] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:52:21.423083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f24ad6-db16-4f84-9fcd-61886d6ba291 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:21.424682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-18f24ad6-db16-4f84-9fcd-61886d6ba291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:21.434245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:21.568963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f24ad6-db16-4f84-9fcd-61886d6ba291 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:21.650387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:21.683912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a6625-0ab8-4585-9fad-a0e9d48d9867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:21.684433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a6625-0ab8-4585-9fad-a0e9d48d9867 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:21.684706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a6625-0ab8-4585-9fad-a0e9d48d9867 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:21.684972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a6625-0ab8-4585-9fad-a0e9d48d9867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:21.687217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1d5a6625-0ab8-4585-9fad-a0e9d48d9867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:21.896640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a6625-0ab8-4585-9fad-a0e9d48d9867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:21.898275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a6625-0ab8-4585-9fad-a0e9d48d9867 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:21.916796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 735/1481] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:24.321340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b265b1-a9bd-4a86-a168-61b605430dee None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:24.321914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b265b1-a9bd-4a86-a168-61b605430dee None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:24.321914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b265b1-a9bd-4a86-a168-61b605430dee None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:24.323222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 747/1482] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:52:24.328881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e538f-108f-45da-a3d6-b4250139c7aa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:24.329360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e538f-108f-45da-a3d6-b4250139c7aa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:24.329466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e538f-108f-45da-a3d6-b4250139c7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:24.400480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b91e538f-108f-45da-a3d6-b4250139c7aa None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:24.401715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b91e538f-108f-45da-a3d6-b4250139c7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:24.452563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:24.511183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 736/1483] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:52:24.803867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b265b1-a9bd-4a86-a168-61b605430dee None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:24.805468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-16b265b1-a9bd-4a86-a168-61b605430dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:24.831986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:24.925993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b265b1-a9bd-4a86-a168-61b605430dee None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:25.005311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80c349-95ef-41be-8372-46d17674be71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:25.005935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80c349-95ef-41be-8372-46d17674be71 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:25.006217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80c349-95ef-41be-8372-46d17674be71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:25.006490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80c349-95ef-41be-8372-46d17674be71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:25.008110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8f80c349-95ef-41be-8372-46d17674be71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:25.087666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:25.124051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80c349-95ef-41be-8372-46d17674be71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:25.125487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80c349-95ef-41be-8372-46d17674be71 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:25.150172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 748/1484] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:27.402741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a538c3-74c8-4415-a567-f9a78b02c4df None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:27.403264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a538c3-74c8-4415-a567-f9a78b02c4df None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:27.403264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a538c3-74c8-4415-a567-f9a78b02c4df None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:27.404575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 737/1485] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:52:27.410139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd37840-dbc0-48b0-a046-a248a276aba5 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:27.410617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd37840-dbc0-48b0-a046-a248a276aba5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:27.410727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd37840-dbc0-48b0-a046-a248a276aba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:27.477115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-bcd37840-dbc0-48b0-a046-a248a276aba5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:27.478357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-bcd37840-dbc0-48b0-a046-a248a276aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:27.579851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 749/1486] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:52:27.875489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a538c3-74c8-4415-a567-f9a78b02c4df None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:27.877025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-47a538c3-74c8-4415-a567-f9a78b02c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:27.982468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a538c3-74c8-4415-a567-f9a78b02c4df None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:28.056169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:28.057968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8daf9-596f-493c-9bf6-0c4fff60cd7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:28.058784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8daf9-596f-493c-9bf6-0c4fff60cd7c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:28.060477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8daf9-596f-493c-9bf6-0c4fff60cd7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:28.061468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8daf9-596f-493c-9bf6-0c4fff60cd7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:28.063267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8fc8daf9-596f-493c-9bf6-0c4fff60cd7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:28.190426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8daf9-596f-493c-9bf6-0c4fff60cd7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:28.191898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8daf9-596f-493c-9bf6-0c4fff60cd7c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:28.211320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 738/1487] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:30.518590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd8833-56ed-4c26-ad18-f452806692d6 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:30.519159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd8833-56ed-4c26-ad18-f452806692d6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:30.519159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd8833-56ed-4c26-ad18-f452806692d6 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:30.520462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 750/1488] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:52:30.526260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a441b-2903-4c36-bd1e-d464fb2cad13 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:30.526754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a441b-2903-4c36-bd1e-d464fb2cad13 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:30.526872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a441b-2903-4c36-bd1e-d464fb2cad13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:30.598158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4b8a441b-2903-4c36-bd1e-d464fb2cad13 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:30.599416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b8a441b-2903-4c36-bd1e-d464fb2cad13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:30.654082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:30.702511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 739/1489] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:52:30.993725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd8833-56ed-4c26-ad18-f452806692d6 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:30.995298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-75bd8833-56ed-4c26-ad18-f452806692d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:31.033627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:31.122780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd8833-56ed-4c26-ad18-f452806692d6 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:31.228076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f9bc-9edf-453b-afcc-c0cd7a678a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:31.228701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f9bc-9edf-453b-afcc-c0cd7a678a94 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:31.228975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f9bc-9edf-453b-afcc-c0cd7a678a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:31.229247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f9bc-9edf-453b-afcc-c0cd7a678a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:31.231031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f101f9bc-9edf-453b-afcc-c0cd7a678a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:31.330025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:31.363609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f9bc-9edf-453b-afcc-c0cd7a678a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:31.365053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f9bc-9edf-453b-afcc-c0cd7a678a94 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:31.385889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 751/1490] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:52:34.009507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc11f-c0da-4eed-b432-e04bd6b493d9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:34.010247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc11f-c0da-4eed-b432-e04bd6b493d9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:34.010247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc11f-c0da-4eed-b432-e04bd6b493d9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:34.011406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 740/1491] 10.222.0.88 () {54 vars in 927 bytes} [Mon Sep 19 11:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:52:34.016786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad7e8-7d43-44fb-9be4-257f8a36e6a9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:34.017246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad7e8-7d43-44fb-9be4-257f8a36e6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:34.017365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad7e8-7d43-44fb-9be4-257f8a36e6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:34.087619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-083ad7e8-7d43-44fb-9be4-257f8a36e6a9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:52:34.088849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-083ad7e8-7d43-44fb-9be4-257f8a36e6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:34.195027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 752/1492] 10.222.0.88 () {58 vars in 1024 bytes} [Mon Sep 19 11:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:52:34.484542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc11f-c0da-4eed-b432-e04bd6b493d9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:34.486101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f1edc11f-c0da-4eed-b432-e04bd6b493d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:34.601073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc11f-c0da-4eed-b432-e04bd6b493d9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:34.681798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014333c-5789-4af1-81f5-0b7ac73cf457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:34.682436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014333c-5789-4af1-81f5-0b7ac73cf457 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:34.682715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014333c-5789-4af1-81f5-0b7ac73cf457 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:34.683014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014333c-5789-4af1-81f5-0b7ac73cf457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:34.684773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8014333c-5789-4af1-81f5-0b7ac73cf457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:34.695364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:34.804642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014333c-5789-4af1-81f5-0b7ac73cf457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:34.806116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014333c-5789-4af1-81f5-0b7ac73cf457 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:34.825581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 741/1493] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:44.193432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=82 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:44.823153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=50 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:49.682220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:49.682825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:49.682825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:49.758828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:53:49.760041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:49.860561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 753/1494] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:53:49.868415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f381ab1-9262-489e-9cb8-21c53497e8ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:49.868903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f381ab1-9262-489e-9cb8-21c53497e8ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:49.869021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f381ab1-9262-489e-9cb8-21c53497e8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:49.943591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3f381ab1-9262-489e-9cb8-21c53497e8ad None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:53:49.944710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3f381ab1-9262-489e-9cb8-21c53497e8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:50.039044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 742/1495] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:53:50.043681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:50.044881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:50.056176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e6abf-a565-4766-9873-9be231715774 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:50.123131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:50.169810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:50.170511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:50.170924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:50.171324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:50.172282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:53:50.172734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:50.174341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:50.174341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:53:50.174341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:53:50.175702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:50.190350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 754/1496] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:53:50.198756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f381ab1-9262-489e-9cb8-21c53497e8ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:50.207880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f381ab1-9262-489e-9cb8-21c53497e8ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:50.268483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:50.301395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:50.302030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:50.302307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:50.302741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:50.303357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:50.305103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:50.305103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 19 11:53:50.305103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: Sep 19 11:53:50.306328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:50.364158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 743/1497] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:53:50 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:53:50.368964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:50.370671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:50.407228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d2e96-92fc-4bd8-ad31-6443fde6a5a9 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:50.507477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:50.507987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:50.508268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:50.508540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:50.509175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:50.510219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:50.510219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:53:50.510219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:53:50.511427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:50.556801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 755/1498] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:53:50 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:53:50.561142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:50.562722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:50.624014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc683e56-ef6c-4078-9286-d13c2b9e2db8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:50.705916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:50.718880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:50.719471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:50.719743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:50.720010 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:50.720672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:50.721691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:50.721691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 19 11:53:50.721691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: Sep 19 11:53:50.722587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:50.737673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 744/1499] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:53:50.741995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:50.743527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:50.754857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0318e5-4ba3-487d-9cb8-117fb6519412 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:50.780446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:50.856628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961ec74-7823-4a90-a541-dcbf640a9e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:50.857854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961ec74-7823-4a90-a541-dcbf640a9e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:50.858130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961ec74-7823-4a90-a541-dcbf640a9e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:50.858395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961ec74-7823-4a90-a541-dcbf640a9e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:50.859069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961ec74-7823-4a90-a541-dcbf640a9e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:50.860060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-a961ec74-7823-4a90-a541-dcbf640a9e70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:50.860060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:53:50.860060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:53:50.860933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961ec74-7823-4a90-a541-dcbf640a9e70 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:50.873203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 756/1500] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:53:50.877372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:50.878906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:50.912375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c59be-8754-484b-afe7-55dd352bbcb1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:51.005971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be21b5-ab30-4031-af74-a33165a6fb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:51.006623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be21b5-ab30-4031-af74-a33165a6fb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:51.006901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be21b5-ab30-4031-af74-a33165a6fb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:51.007172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be21b5-ab30-4031-af74-a33165a6fb6c admin admin] PATH_INFO: `/v3/projects/9789c575183142cd924b2149fb2f956a/users/31e76f4d721f4404b42245ef6233c718/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:51.029071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be21b5-ab30-4031-af74-a33165a6fb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9789c575183142cd924b2149fb2f956a, user_id=31e76f4d721f4404b42245ef6233c718, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:51.030131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-c4be21b5-ab30-4031-af74-a33165a6fb6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:51.030131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 19 11:53:51.030131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: Sep 19 11:53:51.032093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be21b5-ab30-4031-af74-a33165a6fb6c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:51.057119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 745/1501] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:53:50 2022] PUT /identity/v3/projects/9789c575183142cd924b2149fb2f956a/users/31e76f4d721f4404b42245ef6233c718/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:53:51.070135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2230-3bef-4189-83c0-0215fe2a7746 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:51.070427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2230-3bef-4189-83c0-0215fe2a7746 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:51.070704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2230-3bef-4189-83c0-0215fe2a7746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:51.130714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d2ab2230-3bef-4189-83c0-0215fe2a7746 None None] MFA Rules not processed for user `31e76f4d721f4404b42245ef6233c718`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:53:51.131519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d2ab2230-3bef-4189-83c0-0215fe2a7746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:51.209349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 757/1502] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:53:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:53:51.222902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fee09b-f1c7-4251-b9ca-2ee284d17403 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:51.223192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fee09b-f1c7-4251-b9ca-2ee284d17403 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:51.223473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fee09b-f1c7-4251-b9ca-2ee284d17403 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:51.224802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 746/1503] 10.222.0.88 () {54 vars in 967 bytes} [Mon Sep 19 11:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:53:51.229485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d1f59-dfd0-49c1-bb17-3fa1090fe69c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:51.229822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d1f59-dfd0-49c1-bb17-3fa1090fe69c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:51.230107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d1f59-dfd0-49c1-bb17-3fa1090fe69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:51.262867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:51.307948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-cc5d1f59-dfd0-49c1-bb17-3fa1090fe69c None None] MFA Rules not processed for user `37099ac7c39948f0afd272a2b41949ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:53:51.309064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cc5d1f59-dfd0-49c1-bb17-3fa1090fe69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:51.383765 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 758/1504] 10.222.0.88 () {58 vars in 1064 bytes} [Mon Sep 19 11:53:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:53:51.392447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fee09b-f1c7-4251-b9ca-2ee284d17403 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:51.393980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-01fee09b-f1c7-4251-b9ca-2ee284d17403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:51.452055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:51.505829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fee09b-f1c7-4251-b9ca-2ee284d17403 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:51.580394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:51.580933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:51.581206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:51.581483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:51.583115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:51.684165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:51.685627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:51.689137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:51.705157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 747/1505] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:53:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:53:51.752243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f2c76-4d5c-4e3f-a971-672bdd3280f0 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:51.752781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f2c76-4d5c-4e3f-a971-672bdd3280f0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:51.752781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f2c76-4d5c-4e3f-a971-672bdd3280f0 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:51.754071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 759/1506] 10.222.0.88 () {52 vars in 830 bytes} [Mon Sep 19 11:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:53:51.758342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:51.759462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:51.870799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbc51d-1f5f-48cb-8eab-c58e2a11a9fe service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:51.968950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:51.969677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:51.969874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:51.970146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] PATH_INFO: `/v3/users/31e76f4d721f4404b42245ef6233c718` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:51.983352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=31e76f4d721f4404b42245ef6233c718)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:51.984377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:51.984377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 19 11:53:51.984377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: Sep 19 11:53:51.985982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:51.995561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:52.051232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31e76f4d721f4404b42245ef6233c718'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:52.051668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31e76f4d721f4404b42245ef6233c718'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:52.066016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31e76f4d721f4404b42245ef6233c718'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:52.075083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31e76f4d721f4404b42245ef6233c718'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:52.099384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31e76f4d721f4404b42245ef6233c718'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:52.105658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 748/1507] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:53:51 2022] DELETE /identity/v3/users/31e76f4d721f4404b42245ef6233c718 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:53:52.110086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:52.110363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:52.110634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:52.173140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:53:52.174390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:52.250118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:52.263696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 760/1508] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:53:52.273431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:52.274653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:52.380643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64300278-9d07-4705-b140-6c264cd34ffd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:52.435894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:52.456470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450160e-23bf-45cf-a283-3886c65dde3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:52.457078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450160e-23bf-45cf-a283-3886c65dde3b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:52.457356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450160e-23bf-45cf-a283-3886c65dde3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:52.457624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450160e-23bf-45cf-a283-3886c65dde3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:52.459246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2450160e-23bf-45cf-a283-3886c65dde3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:52.577684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450160e-23bf-45cf-a283-3886c65dde3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:52.579146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450160e-23bf-45cf-a283-3886c65dde3b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:52.597177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 749/1509] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:53:52.958530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:52.959786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:52.972489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f8bcb-cbc1-4fcd-a3a7-518bc7d50441 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:53.074655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:53.075308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:53.076159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:53.076431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] PATH_INFO: `/v3/projects/9789c575183142cd924b2149fb2f956a` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:53.087301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9789c575183142cd924b2149fb2f956a)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:53.088316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:53:53.088316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:53:53.088316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:53:53.089506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:53.147702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9789c575183142cd924b2149fb2f956a', 'request_id': 'req-bd72ca08-982f-4454-8933-a7531786c62b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:53.158329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9789c575183142cd924b2149fb2f956a', 'request_id': 'req-bd72ca08-982f-4454-8933-a7531786c62b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:53.172395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9789c575183142cd924b2149fb2f956a', 'request_id': 'req-bd72ca08-982f-4454-8933-a7531786c62b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:53.195191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] The token cache is being invalidate because project 9789c575183142cd924b2149fb2f956a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:53:53.195484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:53.196010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-bd72ca08-982f-4454-8933-a7531786c62b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:53:53.197324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 761/1510] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:53:52 2022] DELETE /identity/v3/projects/9789c575183142cd924b2149fb2f956a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:54:02.594922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=62 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:54:03.193183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:55:26.037896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914759-96fa-4e94-bbce-6925a3f29cfd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:26.038509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914759-96fa-4e94-bbce-6925a3f29cfd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:26.038509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914759-96fa-4e94-bbce-6925a3f29cfd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:26.039776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:26.044645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738e761-a495-47da-8a85-bc4318533dc5 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:26.045133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738e761-a495-47da-8a85-bc4318533dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:26.045244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738e761-a495-47da-8a85-bc4318533dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:26.127460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0738e761-a495-47da-8a85-bc4318533dc5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:26.129338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0738e761-a495-47da-8a85-bc4318533dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:26.233688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:26.388051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914759-96fa-4e94-bbce-6925a3f29cfd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:26.390340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-11914759-96fa-4e94-bbce-6925a3f29cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:26.509699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914759-96fa-4e94-bbce-6925a3f29cfd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:26.515121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be493079-991f-4018-817b-cc4e1cc37429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:26.515722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493079-991f-4018-817b-cc4e1cc37429 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:26.516019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493079-991f-4018-817b-cc4e1cc37429 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:26.516289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493079-991f-4018-817b-cc4e1cc37429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:26.524883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493079-991f-4018-817b-cc4e1cc37429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:26.526490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493079-991f-4018-817b-cc4e1cc37429 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:26.535822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 751/1513] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:27.651918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbc764-26a0-48de-be8d-7c6f2e2de6e1 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:27.652488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbc764-26a0-48de-be8d-7c6f2e2de6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:27.652488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbc764-26a0-48de-be8d-7c6f2e2de6e1 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:27.653952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:27.661165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcadad9-abe6-407e-b08a-4cc69f808034 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:27.661672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcadad9-abe6-407e-b08a-4cc69f808034 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:27.661818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcadad9-abe6-407e-b08a-4cc69f808034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:27.681382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ebcadad9-abe6-407e-b08a-4cc69f808034 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:27.682617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ebcadad9-abe6-407e-b08a-4cc69f808034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:27.690999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:27.734621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f37c8-e72a-4f51-a850-1092732d7d34 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:27.735000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f37c8-e72a-4f51-a850-1092732d7d34 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:27.735351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f37c8-e72a-4f51-a850-1092732d7d34 None None] PATH_INFO: `/v3` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:27.736810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 764/1516] 10.222.0.87 () {54 vars in 925 bytes} [Mon Sep 19 11:55:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:55:27.742760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1c1e-c049-4506-bc6d-6eaf1fadbfa6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:27.743041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1c1e-c049-4506-bc6d-6eaf1fadbfa6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:27.743310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1c1e-c049-4506-bc6d-6eaf1fadbfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:27.804134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-244a1c1e-c049-4506-bc6d-6eaf1fadbfa6 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:27.805336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-244a1c1e-c049-4506-bc6d-6eaf1fadbfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:27.850809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f37c8-e72a-4f51-a850-1092732d7d34 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:27.861199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:55:27.868488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f37c8-e72a-4f51-a850-1092732d7d34 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:27.874055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af342d81-3a33-401a-a5f0-bead3ab6ba1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:27.874665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af342d81-3a33-401a-a5f0-bead3ab6ba1b service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:27.874951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af342d81-3a33-401a-a5f0-bead3ab6ba1b service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:27.875228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af342d81-3a33-401a-a5f0-bead3ab6ba1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:27.881258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 753/1517] 10.222.0.87 () {58 vars in 1015 bytes} [Mon Sep 19 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:27.884356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af342d81-3a33-401a-a5f0-bead3ab6ba1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:27.885984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af342d81-3a33-401a-a5f0-bead3ab6ba1b service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:27.896368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 765/1518] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:55:31.425028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8171397-7fbf-4b50-8159-3973b94f825b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:31.425521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8171397-7fbf-4b50-8159-3973b94f825b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:31.425627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8171397-7fbf-4b50-8159-3973b94f825b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:31.426962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 754/1519] 10.222.0.88 () {54 vars in 961 bytes} [Mon Sep 19 11:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:31.432062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a609d-4d57-49f6-9a46-ea44c5699ab4 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:31.432507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a609d-4d57-49f6-9a46-ea44c5699ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:31.432594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a609d-4d57-49f6-9a46-ea44c5699ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:31.452915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ef5a609d-4d57-49f6-9a46-ea44c5699ab4 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:31.454324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ef5a609d-4d57-49f6-9a46-ea44c5699ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:31.463478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 766/1520] 10.222.0.88 () {58 vars in 1058 bytes} [Mon Sep 19 11:55:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:55:31.471276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8171397-7fbf-4b50-8159-3973b94f825b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:31.484582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8171397-7fbf-4b50-8159-3973b94f825b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:31.490068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06189cd8-530d-4446-a0a5-ade20f6f22a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:31.490661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189cd8-530d-4446-a0a5-ade20f6f22a7 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:31.490943 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189cd8-530d-4446-a0a5-ade20f6f22a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:31.491216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189cd8-530d-4446-a0a5-ade20f6f22a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:31.499924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06189cd8-530d-4446-a0a5-ade20f6f22a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:31.501502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06189cd8-530d-4446-a0a5-ade20f6f22a7 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:31.511661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 755/1521] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:39.454578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86028-cd4e-4073-b71b-5ffd99071eb0 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:39.455117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86028-cd4e-4073-b71b-5ffd99071eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:39.455117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86028-cd4e-4073-b71b-5ffd99071eb0 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:39.456976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:39.463647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8d258-7651-4230-a18d-8bd6e44ba574 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:39.464108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8d258-7651-4230-a18d-8bd6e44ba574 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:39.464231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8d258-7651-4230-a18d-8bd6e44ba574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:39.483793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bee8d258-7651-4230-a18d-8bd6e44ba574 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:39.484980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bee8d258-7651-4230-a18d-8bd6e44ba574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:39.493039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:39.646387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86028-cd4e-4073-b71b-5ffd99071eb0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:39.661602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86028-cd4e-4073-b71b-5ffd99071eb0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:39.667057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd924c3-67c3-4c67-925a-3cea4bffca15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:39.667655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd924c3-67c3-4c67-925a-3cea4bffca15 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:39.667932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd924c3-67c3-4c67-925a-3cea4bffca15 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:39.668204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd924c3-67c3-4c67-925a-3cea4bffca15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:39.676272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd924c3-67c3-4c67-925a-3cea4bffca15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:39.677860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd924c3-67c3-4c67-925a-3cea4bffca15 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:39.687194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 768/1524] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:40.766000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17a148-cd88-499a-a518-b877b5c75adf None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:40.766579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17a148-cd88-499a-a518-b877b5c75adf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:40.766579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17a148-cd88-499a-a518-b877b5c75adf None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:40.767977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:40.774429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ee3dc-04b4-4f82-9df1-df3be3f19ce8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:40.774857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ee3dc-04b4-4f82-9df1-df3be3f19ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:40.774968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ee3dc-04b4-4f82-9df1-df3be3f19ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:40.796330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c68ee3dc-04b4-4f82-9df1-df3be3f19ce8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:40.797515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c68ee3dc-04b4-4f82-9df1-df3be3f19ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:40.805904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:55:40.961898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17a148-cd88-499a-a518-b877b5c75adf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:40.977548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17a148-cd88-499a-a518-b877b5c75adf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:40.983054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cb7da-590e-43ba-8cac-eac778f713a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:40.983646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb7da-590e-43ba-8cac-eac778f713a7 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:40.983922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb7da-590e-43ba-8cac-eac778f713a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:40.984191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb7da-590e-43ba-8cac-eac778f713a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:40.992497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb7da-590e-43ba-8cac-eac778f713a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:40.994122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb7da-590e-43ba-8cac-eac778f713a7 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:41.003390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 758/1527] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:45.127166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731321e-4a36-4c74-a210-18518884d8a3 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:45.127679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731321e-4a36-4c74-a210-18518884d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:45.127679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731321e-4a36-4c74-a210-18518884d8a3 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:45.129497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:45.134855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e808b1-0c1a-425a-b0c7-4df4c6bc6bf9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:45.135294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e808b1-0c1a-425a-b0c7-4df4c6bc6bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:45.135404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e808b1-0c1a-425a-b0c7-4df4c6bc6bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:45.156391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-12e808b1-0c1a-425a-b0c7-4df4c6bc6bf9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:45.157583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12e808b1-0c1a-425a-b0c7-4df4c6bc6bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:45.165326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:45.456421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731321e-4a36-4c74-a210-18518884d8a3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:45.459236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e731321e-4a36-4c74-a210-18518884d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:45.492628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:55:45.538948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731321e-4a36-4c74-a210-18518884d8a3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:45.542716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e5d05-a4da-4df9-a7fb-da730e4a2eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:45.543357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5d05-a4da-4df9-a7fb-da730e4a2eed service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:45.543639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5d05-a4da-4df9-a7fb-da730e4a2eed service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:45.543914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5d05-a4da-4df9-a7fb-da730e4a2eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:45.553884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e5d05-a4da-4df9-a7fb-da730e4a2eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:45.555295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e5d05-a4da-4df9-a7fb-da730e4a2eed service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:45.564630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 771/1530] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:47.730441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cd5b4-5e7a-482a-8e11-0260123217a4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:47.731063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cd5b4-5e7a-482a-8e11-0260123217a4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:47.731063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cd5b4-5e7a-482a-8e11-0260123217a4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:47.732303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:47.737498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02600d1d-d392-47a0-bf07-a2659255f99d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:47.737987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02600d1d-d392-47a0-bf07-a2659255f99d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:47.738133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02600d1d-d392-47a0-bf07-a2659255f99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:47.757152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-02600d1d-d392-47a0-bf07-a2659255f99d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:47.758381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-02600d1d-d392-47a0-bf07-a2659255f99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:47.766153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:48.058797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cd5b4-5e7a-482a-8e11-0260123217a4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:48.060496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e62cd5b4-5e7a-482a-8e11-0260123217a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:48.073797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cd5b4-5e7a-482a-8e11-0260123217a4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:48.077255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80a189-f58c-4751-a17f-a1df4f0831af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:48.078300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80a189-f58c-4751-a17f-a1df4f0831af service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:48.078573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80a189-f58c-4751-a17f-a1df4f0831af service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:48.078839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80a189-f58c-4751-a17f-a1df4f0831af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:48.088520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80a189-f58c-4751-a17f-a1df4f0831af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:48.090076 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80a189-f58c-4751-a17f-a1df4f0831af service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:48.099343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 761/1533] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:51.789740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41938-beb2-44ee-8fd1-95c5ca0d4b0c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:51.790340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41938-beb2-44ee-8fd1-95c5ca0d4b0c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:51.790448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41938-beb2-44ee-8fd1-95c5ca0d4b0c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:51.791765 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:51.797051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ed17f-d499-471b-b603-2f7ef1163c65 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:51.797509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ed17f-d499-471b-b603-2f7ef1163c65 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:51.797610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ed17f-d499-471b-b603-2f7ef1163c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:51.817203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f40ed17f-d499-471b-b603-2f7ef1163c65 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:51.818440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f40ed17f-d499-471b-b603-2f7ef1163c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:51.826236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:52.133666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41938-beb2-44ee-8fd1-95c5ca0d4b0c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:52.135390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-54f41938-beb2-44ee-8fd1-95c5ca0d4b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:52.218111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41938-beb2-44ee-8fd1-95c5ca0d4b0c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:52.221638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87773693-35a6-45a4-bea3-b77727951800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:52.222260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87773693-35a6-45a4-bea3-b77727951800 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:52.222536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87773693-35a6-45a4-bea3-b77727951800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:52.222803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87773693-35a6-45a4-bea3-b77727951800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:52.231622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87773693-35a6-45a4-bea3-b77727951800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:52.233026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87773693-35a6-45a4-bea3-b77727951800 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:52.242709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 774/1536] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:54.433377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9c6f6-573b-428f-84f0-dbc3b3d54999 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:54.434031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9c6f6-573b-428f-84f0-dbc3b3d54999 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:54.434031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9c6f6-573b-428f-84f0-dbc3b3d54999 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:54.435628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:55:54.441521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27203754-c161-4b6b-9948-4bf74f81d4a4 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:54.442436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27203754-c161-4b6b-9948-4bf74f81d4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:54.442436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27203754-c161-4b6b-9948-4bf74f81d4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:54.470490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-27203754-c161-4b6b-9948-4bf74f81d4a4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:55:54.471712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-27203754-c161-4b6b-9948-4bf74f81d4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:54.480509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:55:54.790832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9c6f6-573b-428f-84f0-dbc3b3d54999 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:54.793229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40a9c6f6-573b-428f-84f0-dbc3b3d54999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:54.809989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9c6f6-573b-428f-84f0-dbc3b3d54999 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:54.813592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc11e40-6fda-4741-a243-7577ec78e727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:54.814264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc11e40-6fda-4741-a243-7577ec78e727 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:54.814540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc11e40-6fda-4741-a243-7577ec78e727 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:54.814810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc11e40-6fda-4741-a243-7577ec78e727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:54.830574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc11e40-6fda-4741-a243-7577ec78e727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:54.833541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc11e40-6fda-4741-a243-7577ec78e727 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:54.848494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 764/1539] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:56:04.464785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=63 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:56:04.843161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=37 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:19.902116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b37158-b2be-419d-8963-3894b8cda849 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:19.903205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b37158-b2be-419d-8963-3894b8cda849 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:19.903874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b37158-b2be-419d-8963-3894b8cda849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:19.937699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e0b37158-b2be-419d-8963-3894b8cda849 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:19.939671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e0b37158-b2be-419d-8963-3894b8cda849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:19.955293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 776/1540] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:19.964869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca7a99-df8e-49dd-b1b1-ff5c070bd5e8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:19.966202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca7a99-df8e-49dd-b1b1-ff5c070bd5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:19.966642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca7a99-df8e-49dd-b1b1-ff5c070bd5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:19.993697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0fca7a99-df8e-49dd-b1b1-ff5c070bd5e8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:19.995561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0fca7a99-df8e-49dd-b1b1-ff5c070bd5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.010000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 765/1541] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.014860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b37158-b2be-419d-8963-3894b8cda849 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.031179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b37158-b2be-419d-8963-3894b8cda849 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.041552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.042467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.042922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.043359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.044180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:20.044852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.046890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.059913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 777/1542] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:20.068424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca7a99-df8e-49dd-b1b1-ff5c070bd5e8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.078855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca7a99-df8e-49dd-b1b1-ff5c070bd5e8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.090840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519f93d-45cd-426a-ab3e-914a5594406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.091934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f93d-45cd-426a-ab3e-914a5594406d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.092442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f93d-45cd-426a-ab3e-914a5594406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.093067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f93d-45cd-426a-ab3e-914a5594406d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.093985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519f93d-45cd-426a-ab3e-914a5594406d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.096955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519f93d-45cd-426a-ab3e-914a5594406d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.162597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 766/1543] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:20.168540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.173473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2c2e1-fe5d-42c6-bb75-8c1974757f12 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.173810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2c2e1-fe5d-42c6-bb75-8c1974757f12 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.174121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2c2e1-fe5d-42c6-bb75-8c1974757f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.188198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90379-3dd6-4ebe-9f02-b7a18c2845f5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.222926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9ab2c2e1-fe5d-42c6-bb75-8c1974757f12 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.224164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ab2c2e1-fe5d-42c6-bb75-8c1974757f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.271120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.271744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.272033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.272299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.273005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.275851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.317600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 778/1544] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:20.322280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.353571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 767/1545] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 19 11:57:20.359828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c690a-2f92-4553-9e98-b0199dfa53a5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.360171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c690a-2f92-4553-9e98-b0199dfa53a5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.360400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c690a-2f92-4553-9e98-b0199dfa53a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.372810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1016ad4-2716-4c9d-9f74-75f37f7ab342 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.380017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83a719-87ac-46e4-8159-e83ba3447801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.381533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a719-87ac-46e4-8159-e83ba3447801 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.381848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a719-87ac-46e4-8159-e83ba3447801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.382125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a719-87ac-46e4-8159-e83ba3447801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.382793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83a719-87ac-46e4-8159-e83ba3447801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.384569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83a719-87ac-46e4-8159-e83ba3447801 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.405442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 779/1546] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:57:20.406894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24eafd-db41-4636-9331-38a665325905 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.407190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24eafd-db41-4636-9331-38a665325905 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.407974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24eafd-db41-4636-9331-38a665325905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.418880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-138c690a-2f92-4553-9e98-b0199dfa53a5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.420040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-138c690a-2f92-4553-9e98-b0199dfa53a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.447749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2d24eafd-db41-4636-9331-38a665325905 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.448971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2d24eafd-db41-4636-9331-38a665325905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.451240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 768/1547] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.452491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12f70-4f16-4800-bd5e-831e10efdf96 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.452777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12f70-4f16-4800-bd5e-831e10efdf96 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.453048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12f70-4f16-4800-bd5e-831e10efdf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.469338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 780/1548] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.471490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24eafd-db41-4636-9331-38a665325905 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.477831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-89d12f70-4f16-4800-bd5e-831e10efdf96 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.479047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-89d12f70-4f16-4800-bd5e-831e10efdf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.482828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24eafd-db41-4636-9331-38a665325905 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.488598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.489047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 769/1549] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.489217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.489484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.489757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.490140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f9abd-8b4b-4f23-9da0-774e2300f5e4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.490415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f9abd-8b4b-4f23-9da0-774e2300f5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.490577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.490745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f9abd-8b4b-4f23-9da0-774e2300f5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.495412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.510726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 781/1550] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:20.511696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.521868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-211f9abd-8b4b-4f23-9da0-774e2300f5e4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.522747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd4114-00e0-46c1-aa11-19352a3824e3 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.523119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-211f9abd-8b4b-4f23-9da0-774e2300f5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.534487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c6a24-590b-470e-8f7a-68e2987e1a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.535207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c6a24-590b-470e-8f7a-68e2987e1a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.535489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c6a24-590b-470e-8f7a-68e2987e1a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.535761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c6a24-590b-470e-8f7a-68e2987e1a64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.536423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c6a24-590b-470e-8f7a-68e2987e1a64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:20.537076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c6a24-590b-470e-8f7a-68e2987e1a64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.539171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c6a24-590b-470e-8f7a-68e2987e1a64 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.541271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 770/1551] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.542760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7269c6-2d45-42e3-b7e2-9cd2b1b50c1e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.543040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7269c6-2d45-42e3-b7e2-9cd2b1b50c1e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.543315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7269c6-2d45-42e3-b7e2-9cd2b1b50c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.556701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 782/1552] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:20.558358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b819a5-38cf-4176-8cb0-ead636075809 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.558642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b819a5-38cf-4176-8cb0-ead636075809 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.558915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b819a5-38cf-4176-8cb0-ead636075809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.564427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bd7269c6-2d45-42e3-b7e2-9cd2b1b50c1e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.566034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd7269c6-2d45-42e3-b7e2-9cd2b1b50c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.574370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 771/1553] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.575497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5bc05-22a2-4cf2-9f6f-12f6f3d18a43 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.575787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5bc05-22a2-4cf2-9f6f-12f6f3d18a43 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.576060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5bc05-22a2-4cf2-9f6f-12f6f3d18a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.583109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-38b819a5-38cf-4176-8cb0-ead636075809 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.584310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-38b819a5-38cf-4176-8cb0-ead636075809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.595970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 783/1554] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.596820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b819a5-38cf-4176-8cb0-ead636075809 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.600510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-94e5bc05-22a2-4cf2-9f6f-12f6f3d18a43 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.601711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94e5bc05-22a2-4cf2-9f6f-12f6f3d18a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.606657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b819a5-38cf-4176-8cb0-ead636075809 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.609791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 772/1555] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.611312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149eb54-c25f-407c-977f-c8cdc4b3603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.611972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149eb54-c25f-407c-977f-c8cdc4b3603f admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.612251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149eb54-c25f-407c-977f-c8cdc4b3603f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.612545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149eb54-c25f-407c-977f-c8cdc4b3603f admin admin] PATH_INFO: `/v3/projects/02d3b2d4d255495f91e42bf4c39a41a7/users/f6e2d26fb24a4ec88cd54088f1150e96/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.614560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd727f-fd1d-448c-b053-0bf9a08b04ba None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.614853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd727f-fd1d-448c-b053-0bf9a08b04ba None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.615132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd727f-fd1d-448c-b053-0bf9a08b04ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.628307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149eb54-c25f-407c-977f-c8cdc4b3603f admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d3b2d4d255495f91e42bf4c39a41a7, user_id=f6e2d26fb24a4ec88cd54088f1150e96, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.629340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-2149eb54-c25f-407c-977f-c8cdc4b3603f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:57:20.629340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:57:20.629340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:57:20.631296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149eb54-c25f-407c-977f-c8cdc4b3603f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.635957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a6fd727f-fd1d-448c-b053-0bf9a08b04ba None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.637606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6fd727f-fd1d-448c-b053-0bf9a08b04ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.645780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 784/1556] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:20 2022] PUT /identity/v3/projects/02d3b2d4d255495f91e42bf4c39a41a7/users/f6e2d26fb24a4ec88cd54088f1150e96/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:20.647014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb372d-d7e8-4a4f-a2d7-e99fb6b5986c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.647375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb372d-d7e8-4a4f-a2d7-e99fb6b5986c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.647569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb372d-d7e8-4a4f-a2d7-e99fb6b5986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.667384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8dbb372d-d7e8-4a4f-a2d7-e99fb6b5986c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.668585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8dbb372d-d7e8-4a4f-a2d7-e99fb6b5986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.700300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 773/1557] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.701477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422eff9-1a09-4446-9c15-aa26f277366a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.701787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422eff9-1a09-4446-9c15-aa26f277366a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.702067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422eff9-1a09-4446-9c15-aa26f277366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.721050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0422eff9-1a09-4446-9c15-aa26f277366a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.722293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0422eff9-1a09-4446-9c15-aa26f277366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.724025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:20.731007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 774/1558] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.732060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422eff9-1a09-4446-9c15-aa26f277366a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.734413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 785/1559] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.736188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f30c35-7087-4ca9-ab89-cbb70e29d8fa None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.736474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f30c35-7087-4ca9-ab89-cbb70e29d8fa None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.736756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f30c35-7087-4ca9-ab89-cbb70e29d8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.742577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422eff9-1a09-4446-9c15-aa26f277366a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.747198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.747800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.748083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.748357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.748989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.750924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.756871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-f1f30c35-7087-4ca9-ab89-cbb70e29d8fa None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.758113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f1f30c35-7087-4ca9-ab89-cbb70e29d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.767576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 786/1560] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.768381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f30c35-7087-4ca9-ab89-cbb70e29d8fa None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.778293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f30c35-7087-4ca9-ab89-cbb70e29d8fa None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.784063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219839d7-8324-463d-83e8-843e040e2190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.784685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219839d7-8324-463d-83e8-843e040e2190 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.784966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219839d7-8324-463d-83e8-843e040e2190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.785304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219839d7-8324-463d-83e8-843e040e2190 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.786023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219839d7-8324-463d-83e8-843e040e2190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:20.786330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219839d7-8324-463d-83e8-843e040e2190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.786891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 775/1561] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:20.787690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.787959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219839d7-8324-463d-83e8-843e040e2190 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.794567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:20.802928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 787/1562] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:20.804208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686278-cfaa-4357-b4ae-82cff81659d6 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.804497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686278-cfaa-4357-b4ae-82cff81659d6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.804779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686278-cfaa-4357-b4ae-82cff81659d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.836652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b8686278-cfaa-4357-b4ae-82cff81659d6 None None] MFA Rules not processed for user `f6e2d26fb24a4ec88cd54088f1150e96`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.837895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b8686278-cfaa-4357-b4ae-82cff81659d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.856065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20659b-9ffe-4741-b9d6-744d6634f676 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.861319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.861958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.862237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.862506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.863148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:20.863451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.864462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:57:20.864462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 19 11:57:20.864462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: Sep 19 11:57:20.865342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd78d3-549e-40fb-a580-601917649608 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.877883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 776/1563] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:20.879251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1d29-70dd-4621-9e5c-14c3b5a36ae0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.879641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1d29-70dd-4621-9e5c-14c3b5a36ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.880037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1d29-70dd-4621-9e5c-14c3b5a36ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.899741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-79bb1d29-70dd-4621-9e5c-14c3b5a36ae0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.900963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-79bb1d29-70dd-4621-9e5c-14c3b5a36ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.907930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 788/1564] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.912845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686278-cfaa-4357-b4ae-82cff81659d6 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.912946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 777/1565] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.912946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff1db9-074d-46a1-9d90-2c8f8d2107f4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.912946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff1db9-074d-46a1-9d90-2c8f8d2107f4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.912946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff1db9-074d-46a1-9d90-2c8f8d2107f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.924838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686278-cfaa-4357-b4ae-82cff81659d6 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.930010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.930623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.930905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.931180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.931795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-12ff1db9-074d-46a1-9d90-2c8f8d2107f4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:20.931908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:20.932187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.932991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12ff1db9-074d-46a1-9d90-2c8f8d2107f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.933874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.942769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 778/1566] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:20.946809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 789/1567] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:20.947557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.954416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff1db9-074d-46a1-9d90-2c8f8d2107f4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.958283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de177-1738-4d44-9222-13e29d7db92b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.963971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.964586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.964868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.965042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff1db9-074d-46a1-9d90-2c8f8d2107f4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.965551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.965754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.967687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.969723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.970385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.970667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.970939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.971580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:20.971883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.973495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.988335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 779/1568] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:20.989240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.995062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 790/1569] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/users => generated 283 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:20.995062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.000903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4e909-3cb7-44ec-83fe-77799f94f180 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.006360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.006736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37cb20-b92e-4e06-90b1-5b6263b5031e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.006971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.007253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.007528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.008506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.010443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.011863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e90e4-0094-44ac-a244-0fa61c474c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.012520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e90e4-0094-44ac-a244-0fa61c474c23 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.012829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e90e4-0094-44ac-a244-0fa61c474c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.013101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e90e4-0094-44ac-a244-0fa61c474c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.013729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e90e4-0094-44ac-a244-0fa61c474c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.014912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-433e90e4-0094-44ac-a244-0fa61c474c23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:57:21.014912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:57:21.014912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:57:21.015944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e90e4-0094-44ac-a244-0fa61c474c23 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.040173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 780/1570] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.041259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.043758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 791/1571] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:20 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.045056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2028-2f24-45af-81a5-bb21ef8e0435 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.045345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2028-2f24-45af-81a5-bb21ef8e0435 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.045625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2028-2f24-45af-81a5-bb21ef8e0435 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.047293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 792/1572] 10.222.0.88 () {54 vars in 965 bytes} [Mon Sep 19 11:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:57:21.047997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf2028-2f24-45af-81a5-bb21ef8e0435 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.107510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e8fff-5ce6-4995-b1ce-37079358f59c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.112719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.113319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.113595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.113901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.114343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf2028-2f24-45af-81a5-bb21ef8e0435 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.114533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:21.114835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.116432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.119012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.119613 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.119882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.120152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.120788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:21.121087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.122677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.130003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 781/1573] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:21.130003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.140898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 793/1574] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:21.141899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.150394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270509e0-7ad5-47db-acf3-c627e7206adf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.155030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.155634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.155953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.156080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3e1fe-d2b3-4474-8817-de1ff1cc8e2b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.156227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.156911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.159441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.161300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.161938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.162231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.162510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.163182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.164769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.167023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:21.177220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 794/1575] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.178139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.189297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434058-6b13-41de-bb70-c045c2ec0042 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.193957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 782/1576] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.193957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.216796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:21.250054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.250705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.251022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.251508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.252266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.254812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.263717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab4783-bd35-47ce-b1a5-5c64ec1cd542 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.268798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0550c16-454b-4cc7-ab4e-ff1372455c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.269470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0550c16-454b-4cc7-ab4e-ff1372455c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.269749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0550c16-454b-4cc7-ab4e-ff1372455c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.270198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0550c16-454b-4cc7-ab4e-ff1372455c7d admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.270937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0550c16-454b-4cc7-ab4e-ff1372455c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.271949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-b0550c16-454b-4cc7-ab4e-ff1372455c7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:57:21.271949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 19 11:57:21.271949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: Sep 19 11:57:21.273143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0550c16-454b-4cc7-ab4e-ff1372455c7d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.285719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 795/1577] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.286561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.299115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 783/1578] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.300525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8b8d4-cf0d-442d-af33-54287f8c737f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.300805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8b8d4-cf0d-442d-af33-54287f8c737f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.301078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8b8d4-cf0d-442d-af33-54287f8c737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.339933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f5e8b8d4-cf0d-442d-af33-54287f8c737f None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:21.341276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f5e8b8d4-cf0d-442d-af33-54287f8c737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:21.353552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89b377-6ecf-4294-bf9f-8418bd34a902 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.359025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.359649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.359928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.360243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.361669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.364250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.387351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 796/1579] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.389031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.401327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3f91-dbd7-4e44-b242-76de9e51e711 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.407116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.408340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.408791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.409152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.410090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.412217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.419615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 784/1580] 10.222.0.88 () {58 vars in 1062 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:21.420513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8b8d4-cf0d-442d-af33-54287f8c737f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.434731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8b8d4-cf0d-442d-af33-54287f8c737f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.439297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.439849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.440123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.440390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.441014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.441958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 797/1581] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.443079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.446023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.472105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 785/1582] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.476230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.499250 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:21.512791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b71f31-85d2-4324-93a4-a3868b42d553 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.549031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bd4bc-6be2-49ad-bb0d-7cf998af731c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.553744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.554381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.554662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.554929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.555644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.557448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.569212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:21.584501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 786/1583] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.585326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.589432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.590354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.590936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.591445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.592561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.594525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.595891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbb987-a16c-47cc-8878-f8fc6e7e4952 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.601632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.602303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.602586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.603008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.603668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.606098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.607925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 798/1584] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.609433 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.621110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ecc1-5648-4a24-b305-d776b1748b02 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.626871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.627739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.628171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.628467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.629229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.630734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.635469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 787/1585] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/users => generated 295 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.636631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.646494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 799/1586] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.647316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.652213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd293690-11a0-4dbd-99f2-aee5e35a852c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.658662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.659321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.659561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.659831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.660484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.662104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.674823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 788/1587] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.675657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.685995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1465af-5d8a-4f74-9f97-90959ff55a68 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.691250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.691862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.692144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.692434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.693064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.695110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.720320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 789/1588] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.721384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.726491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc813146-66d6-4e59-91f6-dc46186ba04b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.731933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.732539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.732814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.733083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.733470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6020c-4a72-4461-b761-a3ade7772712 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.742698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.743410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.744188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.744188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.744582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.749748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.775081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 790/1589] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:21.776022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.787964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ffbfd-5845-48ce-a3b1-0c7c5af28122 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.798961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.799819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.800172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.800451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.801112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.803066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.814365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.815949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.816189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 791/1590] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.817020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.827061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 800/1591] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:21.827819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.828112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88377000-7217-4cb1-8d3a-58cd53662016 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.833268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.834250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.834387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.834584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] PATH_INFO: `/v3/projects/572b24f6b830413ca03007b0eb1c7695/users/4ece529671e94fc6a9beba42631d2bd3/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.838732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc4881-11aa-4097-9925-1893b007a082 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.843455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.844153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.844527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.846091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.846242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.847820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.856395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=572b24f6b830413ca03007b0eb1c7695, user_id=4ece529671e94fc6a9beba42631d2bd3, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.858794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 801/1592] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.859566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.863590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.872513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce5b73-9230-4401-956d-04dbc2f91a6a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.879443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:21.880659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 792/1593] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:21 2022] PUT /identity/v3/projects/572b24f6b830413ca03007b0eb1c7695/users/4ece529671e94fc6a9beba42631d2bd3/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:21.881855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.953192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.953856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.954114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.954391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.955039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.956626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.958216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647055f-7df1-479a-82e5-97931b4cbfc6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.963028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.963969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.964284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.964568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.965159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.966814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.971543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 802/1594] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.971543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.979508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb7f0-99ba-45a9-afdb-e3ef8d1a187b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.985625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 793/1595] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.985625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.988119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.988725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.989017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.989292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.990093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.991730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.997736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352f6c6-9743-4a8b-b4e5-68e0e52aa60b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.003351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 803/1596] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:22.004165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.006049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.006946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.007237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.007514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.008273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.010931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.013922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3d834-ff42-46bc-912b-aac8c5a4f13d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.018688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a9b7-e9a8-4e83-a8e0-d8f1bf72d66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.019270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a9b7-e9a8-4e83-a8e0-d8f1bf72d66f admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.019553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a9b7-e9a8-4e83-a8e0-d8f1bf72d66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.019825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a9b7-e9a8-4e83-a8e0-d8f1bf72d66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.020454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a9b7-e9a8-4e83-a8e0-d8f1bf72d66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.022072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 794/1597] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:22.022700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.025074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a9b7-e9a8-4e83-a8e0-d8f1bf72d66f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.033351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b527f-e59f-495f-aa95-729ab4edbae7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.039531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 804/1598] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:22.039531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760b332-7272-4702-95bf-d214a041ce66 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.039531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760b332-7272-4702-95bf-d214a041ce66 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.039531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760b332-7272-4702-95bf-d214a041ce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.041688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.042570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.042819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.043036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.043677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.045273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.057169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 795/1599] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:22.058514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.071076 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a1e30-0107-42c3-92ec-e56fc02d5465 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.071337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8760b332-7272-4702-95bf-d214a041ce66 None None] MFA Rules not processed for user `4ece529671e94fc6a9beba42631d2bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.072634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8760b332-7272-4702-95bf-d214a041ce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.076288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.077227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.077439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.077716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] PATH_INFO: `/v3/projects/ced3cefcaf434541a38dd01270076279/users/1015b24d41f446069b0e0884b92a1772/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.095683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced3cefcaf434541a38dd01270076279, user_id=1015b24d41f446069b0e0884b92a1772, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.098709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.112447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 796/1600] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:22 2022] PUT /identity/v3/projects/ced3cefcaf434541a38dd01270076279/users/1015b24d41f446069b0e0884b92a1772/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:22.113232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.121733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:22.147298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 805/1601] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.148157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760b332-7272-4702-95bf-d214a041ce66 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.159572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760b332-7272-4702-95bf-d214a041ce66 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.188278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451ff3e-0993-44af-acf9-9d8320657cc4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.194166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.194166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.194416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.194670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] PATH_INFO: `/v3/projects/bf7ef0ac87cb450f9320b8d3a313d942/users/11af9e7e3fed4bea9187dee0ebf27551/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.211001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7ef0ac87cb450f9320b8d3a313d942, user_id=11af9e7e3fed4bea9187dee0ebf27551, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.214128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.218075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.218730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.219076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.219372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.220290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.222086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.228523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 797/1602] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:22 2022] PUT /identity/v3/projects/bf7ef0ac87cb450f9320b8d3a313d942/users/11af9e7e3fed4bea9187dee0ebf27551/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:22.232705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.237567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 806/1603] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:22.237567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.245386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7c5b-521c-4299-ba5a-277e2b20b5d8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.250500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:22.295925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c0e50-44cc-4e5e-b742-a35dbb4daa49 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.301559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.302246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.302532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.302797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.303432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.304990 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691c86e-072b-44ef-938d-b457738d8799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.305132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.305642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691c86e-072b-44ef-938d-b457738d8799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.305955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691c86e-072b-44ef-938d-b457738d8799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.306294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691c86e-072b-44ef-938d-b457738d8799 admin admin] PATH_INFO: `/v3/projects/5c3286ac6e9d49ebaba04fcb5d579855/users/ae4ccb010348465e9adf93d156f98fe8/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.315930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 798/1604] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:22.316733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.323011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691c86e-072b-44ef-938d-b457738d8799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3286ac6e9d49ebaba04fcb5d579855, user_id=ae4ccb010348465e9adf93d156f98fe8, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.325735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691c86e-072b-44ef-938d-b457738d8799 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.326255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78faec87-4ada-46a7-b2c8-c8a5a9e6ef4e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.332475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.333147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.333442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.334009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] PATH_INFO: `/v3/projects/256ccce56c534cacb247aa128399f3f1/users/766511b86ad643c099998fc4e1638f9a/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.341406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 807/1605] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:22 2022] PUT /identity/v3/projects/5c3286ac6e9d49ebaba04fcb5d579855/users/ae4ccb010348465e9adf93d156f98fe8/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:22.342933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6b73-ef9c-45da-9974-15014647c111 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.343339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6b73-ef9c-45da-9974-15014647c111 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.343710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6b73-ef9c-45da-9974-15014647c111 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.345786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 808/1606] 10.222.0.88 () {54 vars in 912 bytes} [Mon Sep 19 11:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:57:22.348422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5581-bab6-4731-a18f-28d234a64457 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.349370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5581-bab6-4731-a18f-28d234a64457 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.349720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5581-bab6-4731-a18f-28d234a64457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.352436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256ccce56c534cacb247aa128399f3f1, user_id=766511b86ad643c099998fc4e1638f9a, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.355270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.370708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d49e5581-bab6-4731-a18f-28d234a64457 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.371908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d49e5581-bab6-4731-a18f-28d234a64457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.372304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 799/1607] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:22 2022] PUT /identity/v3/projects/256ccce56c534cacb247aa128399f3f1/users/766511b86ad643c099998fc4e1638f9a/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:22.373149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.435680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1258003-4984-4566-9cca-3982308b4c55 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.439826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.440429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.440764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.441116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.442115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 809/1608] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.443357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b6179-2fbb-41fe-a35e-dadf3926cab5 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.443641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b6179-2fbb-41fe-a35e-dadf3926cab5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.443914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b6179-2fbb-41fe-a35e-dadf3926cab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.457884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:22.464265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-841b6179-2fbb-41fe-a35e-dadf3926cab5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.466151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-841b6179-2fbb-41fe-a35e-dadf3926cab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.473436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 810/1609] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.474265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b6179-2fbb-41fe-a35e-dadf3926cab5 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.484036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b6179-2fbb-41fe-a35e-dadf3926cab5 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.489072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412b60a-e3c8-4242-a788-ea83000dde74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.489724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b60a-e3c8-4242-a788-ea83000dde74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.490036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b60a-e3c8-4242-a788-ea83000dde74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.490310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b60a-e3c8-4242-a788-ea83000dde74 admin admin] PATH_INFO: `/v3/projects/ec6875fa35a4462698c06020a0238285/users/8fa4b6af47384c7fb26aae3547d02b64/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.505284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412b60a-e3c8-4242-a788-ea83000dde74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6875fa35a4462698c06020a0238285, user_id=8fa4b6af47384c7fb26aae3547d02b64, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.507478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.507981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412b60a-e3c8-4242-a788-ea83000dde74 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.508902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.518405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 800/1610] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:22.519195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.523818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 811/1611] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:22 2022] PUT /identity/v3/projects/ec6875fa35a4462698c06020a0238285/users/8fa4b6af47384c7fb26aae3547d02b64/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:22.525741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72757742-cd02-4ce6-baa3-66f8442a4c23 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.526225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72757742-cd02-4ce6-baa3-66f8442a4c23 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.526682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72757742-cd02-4ce6-baa3-66f8442a4c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.534633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df20656-dc1a-49bb-9330-190079765530 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.559097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-72757742-cd02-4ce6-baa3-66f8442a4c23 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.561009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-72757742-cd02-4ce6-baa3-66f8442a4c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.609213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f078290-e09e-4dab-b654-5dd1a0c87680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.610299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f078290-e09e-4dab-b654-5dd1a0c87680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.611400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f078290-e09e-4dab-b654-5dd1a0c87680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.611907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f078290-e09e-4dab-b654-5dd1a0c87680 admin admin] PATH_INFO: `/v3/projects/2bbb869d1a69445f8622d3b7941f4b8e/users/312e4f8bc2164c4bbca79e5186726594/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.632537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f078290-e09e-4dab-b654-5dd1a0c87680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bbb869d1a69445f8622d3b7941f4b8e, user_id=312e4f8bc2164c4bbca79e5186726594, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.635341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f078290-e09e-4dab-b654-5dd1a0c87680 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.664137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 801/1612] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:22 2022] PUT /identity/v3/projects/2bbb869d1a69445f8622d3b7941f4b8e/users/312e4f8bc2164c4bbca79e5186726594/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:22.664137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d685de3-b6ee-4576-a055-b09cbf51c0f6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.664137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d685de3-b6ee-4576-a055-b09cbf51c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.664137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d685de3-b6ee-4576-a055-b09cbf51c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.664809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:22.680421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 812/1613] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.682387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84316f7-460c-4aa4-96c8-946cee7d5fdb None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.682839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84316f7-460c-4aa4-96c8-946cee7d5fdb None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.683271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84316f7-460c-4aa4-96c8-946cee7d5fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.702620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3d685de3-b6ee-4576-a055-b09cbf51c0f6 None None] MFA Rules not processed for user `ae4ccb010348465e9adf93d156f98fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.704001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3d685de3-b6ee-4576-a055-b09cbf51c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.704877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d84316f7-460c-4aa4-96c8-946cee7d5fdb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.706082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d84316f7-460c-4aa4-96c8-946cee7d5fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.779669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 813/1614] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.780601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84316f7-460c-4aa4-96c8-946cee7d5fdb None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.786981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 802/1615] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.788388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d685de3-b6ee-4576-a055-b09cbf51c0f6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.827344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:22.861604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d685de3-b6ee-4576-a055-b09cbf51c0f6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.866342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10535-9d05-4d60-af2e-6633679429ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.866977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10535-9d05-4d60-af2e-6633679429ba service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.867254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10535-9d05-4d60-af2e-6633679429ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.867526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10535-9d05-4d60-af2e-6633679429ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.870680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84316f7-460c-4aa4-96c8-946cee7d5fdb None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.877691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f171b5-ffd7-4a82-9033-a92753b3425d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.878402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f171b5-ffd7-4a82-9033-a92753b3425d service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.878689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f171b5-ffd7-4a82-9033-a92753b3425d service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.878966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f171b5-ffd7-4a82-9033-a92753b3425d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.879598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10535-9d05-4d60-af2e-6633679429ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.881260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10535-9d05-4d60-af2e-6633679429ba service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.889361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f171b5-ffd7-4a82-9033-a92753b3425d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.890903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f171b5-ffd7-4a82-9033-a92753b3425d service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.894042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 803/1616] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:22.895427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcdee0-745d-4aab-a436-d171bb1e421d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.895720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcdee0-745d-4aab-a436-d171bb1e421d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.896020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcdee0-745d-4aab-a436-d171bb1e421d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.904578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 814/1617] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:22.905603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0205-3091-4b05-a5b7-47beb376d4a2 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.905917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0205-3091-4b05-a5b7-47beb376d4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.906199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0205-3091-4b05-a5b7-47beb376d4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.922856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ebdcdee0-745d-4aab-a436-d171bb1e421d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.925235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ebdcdee0-745d-4aab-a436-d171bb1e421d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.937660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a8fb0205-3091-4b05-a5b7-47beb376d4a2 None None] MFA Rules not processed for user `312e4f8bc2164c4bbca79e5186726594`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:22.939053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 804/1618] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.939718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a8fb0205-3091-4b05-a5b7-47beb376d4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.940167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcdee0-745d-4aab-a436-d171bb1e421d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.020392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcdee0-745d-4aab-a436-d171bb1e421d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.026547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.027583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.028189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.028701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.040719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 815/1619] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:23.041699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb0205-3091-4b05-a5b7-47beb376d4a2 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.057540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb0205-3091-4b05-a5b7-47beb376d4a2 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.064347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.065335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.065688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.066036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.067286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.070073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.123538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.125637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.146536 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 816/1620] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:23 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:23.147116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.149865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 805/1621] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:23.153024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.160626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:23.165141 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add33820-3a35-47d1-8d4f-e8fd076007fd service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.281619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.282279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b86cf-54c5-4b90-be82-d067f10baf66 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.283985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.284469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.284907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.292960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.293281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.293394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.293633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.339420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:23.426467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.431017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.443057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.444800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.455027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 817/1622] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:23.456277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.461440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 806/1623] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:23.462765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.476037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7cacb-3c91-483d-813c-2a770575eefc service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.481616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.482826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.483338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.483825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.483936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5adab0-0942-44a1-943c-520edcbba2e3 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.484766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.487777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.502335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.503018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.503430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.503714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.504475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.506786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.527152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 807/1624] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:23 2022] POST /identity/v3/users => generated 291 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:23.532372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.550965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca8a8c-09ed-440d-b626-9599e4c829cc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.559510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.560619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.561146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.561744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.562900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.566302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.578514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 818/1625] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:23 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:23.583546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.584603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 808/1626] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:23.594078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.623848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd270b-cad3-496a-bbba-74c86b44c09f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.674293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a0928-708b-4afd-a8cc-99b8f6677a55 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.681945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.682692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.683142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.683451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.684351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.686959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.727974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 819/1627] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:23 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:23.734086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.755798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.756534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.756665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb3d98-8fdf-47ec-af3a-aa5d7f2fd730 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.757121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.758426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] PATH_INFO: `/v3/projects/b4d7da9c939742299a69303da69e0b66/users/523e101548424731ada32b29c091afa2/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.773570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:23.776220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:23.776728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:23.777293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:23.779470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.784941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.808142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4d7da9c939742299a69303da69e0b66, user_id=523e101548424731ada32b29c091afa2, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:23.814757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:23.815919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 820/1628] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:23.818359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.841331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 809/1629] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:23 2022] PUT /identity/v3/projects/b4d7da9c939742299a69303da69e0b66/users/523e101548424731ada32b29c091afa2/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:23.842920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:23.847377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb399a80-5800-477e-b2f6-50ef595e17a7 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:23.874617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:24.039165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:24.041050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedaf12-6636-4aa0-a552-3887d8ddc88b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:24.041504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.042184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.043467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] PATH_INFO: `/v3/projects/4310b33484fc4e0aa04304a682e95f5e/users/086cd4da730f4e77b7a4161f5719ae40/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.060071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:24.060669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.060956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.061234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.062012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:24.064317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:24.085994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4310b33484fc4e0aa04304a682e95f5e, user_id=086cd4da730f4e77b7a4161f5719ae40, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:24.096502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 810/1630] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:24.106834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:24.117585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:24.120834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8377ce-75c1-413f-b413-c79b6d30f9c1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:24.134738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1716100-3963-4a56-ab51-4f094b97ed9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:24.135591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1716100-3963-4a56-ab51-4f094b97ed9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.137280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1716100-3963-4a56-ab51-4f094b97ed9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.137854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1716100-3963-4a56-ab51-4f094b97ed9a admin admin] PATH_INFO: `/v3/projects/b4d7da9c939742299a69303da69e0b66/users/523e101548424731ada32b29c091afa2/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.141390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1716100-3963-4a56-ab51-4f094b97ed9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4d7da9c939742299a69303da69e0b66, user_id=523e101548424731ada32b29c091afa2, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:24.148247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 821/1631] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:23 2022] PUT /identity/v3/projects/4310b33484fc4e0aa04304a682e95f5e/users/086cd4da730f4e77b7a4161f5719ae40/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:24.150295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1716100-3963-4a56-ab51-4f094b97ed9a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:24.153476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:24.155176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:24.173631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 811/1632] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:24 2022] PUT /identity/v3/projects/b4d7da9c939742299a69303da69e0b66/users/523e101548424731ada32b29c091afa2/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:24.183904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5f2a2-ca9c-4898-955d-9f23bd99245c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.184786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5f2a2-ca9c-4898-955d-9f23bd99245c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.185283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5f2a2-ca9c-4898-955d-9f23bd99245c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.250258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf1142-a484-461d-b800-f3cbdd893ee3 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:24.273649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-44b5f2a2-ca9c-4898-955d-9f23bd99245c None None] MFA Rules not processed for user `523e101548424731ada32b29c091afa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:24.275018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44b5f2a2-ca9c-4898-955d-9f23bd99245c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:24.358701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:24.359568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.360319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.361542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.362388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:24.365715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:24.392900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 822/1633] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:24.395809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:24.419384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a0912-197b-42d7-aa39-f3a83a606469 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:24.428567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 812/1634] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:24.433496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a703aa7-0226-4ffe-93c0-b7188ed9f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:24.434336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703aa7-0226-4ffe-93c0-b7188ed9f0db admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.434709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703aa7-0226-4ffe-93c0-b7188ed9f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.435098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703aa7-0226-4ffe-93c0-b7188ed9f0db admin admin] PATH_INFO: `/v3/projects/4310b33484fc4e0aa04304a682e95f5e/users/086cd4da730f4e77b7a4161f5719ae40/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.445488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a703aa7-0226-4ffe-93c0-b7188ed9f0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=4310b33484fc4e0aa04304a682e95f5e, user_id=086cd4da730f4e77b7a4161f5719ae40, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:24.445844 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5f2a2-ca9c-4898-955d-9f23bd99245c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:24.445970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a703aa7-0226-4ffe-93c0-b7188ed9f0db admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:24.479053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 823/1635] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:24 2022] PUT /identity/v3/projects/4310b33484fc4e0aa04304a682e95f5e/users/086cd4da730f4e77b7a4161f5719ae40/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:24.489088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac544-13cd-4dd0-9582-f2cda9607625 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.489532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac544-13cd-4dd0-9582-f2cda9607625 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.490124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac544-13cd-4dd0-9582-f2cda9607625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.552845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-19bac544-13cd-4dd0-9582-f2cda9607625 None None] MFA Rules not processed for user `086cd4da730f4e77b7a4161f5719ae40`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:24.556863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-19bac544-13cd-4dd0-9582-f2cda9607625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:24.564083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5f2a2-ca9c-4898-955d-9f23bd99245c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:24.587692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:24.683260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:24.684521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.686561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.687114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.729270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 824/1636] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:24.759267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facabb4a-8c54-4a6e-8235-76bb46dcaaf8 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.759719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facabb4a-8c54-4a6e-8235-76bb46dcaaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.760317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facabb4a-8c54-4a6e-8235-76bb46dcaaf8 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.762206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 825/1637] 10.222.0.88 () {54 vars in 965 bytes} [Mon Sep 19 11:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:57:24.772897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:24.776349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92fbb-7ce1-4144-9038-ddcaea8318b9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.777145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92fbb-7ce1-4144-9038-ddcaea8318b9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.778203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92fbb-7ce1-4144-9038-ddcaea8318b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.811320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:24.813041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:24.821576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c4f92fbb-7ce1-4144-9038-ddcaea8318b9 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:24.824693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c4f92fbb-7ce1-4144-9038-ddcaea8318b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:24.839837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 813/1638] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:57:24.840845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:24.857195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318cfb-d1a6-42f9-99e1-8a278c28f270 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:24.871977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2edc89-4a96-4c76-9ddb-49f332b1dd9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:24.873230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2edc89-4a96-4c76-9ddb-49f332b1dd9e service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.873230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2edc89-4a96-4c76-9ddb-49f332b1dd9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.873473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2edc89-4a96-4c76-9ddb-49f332b1dd9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.889197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2edc89-4a96-4c76-9ddb-49f332b1dd9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:24.892573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2edc89-4a96-4c76-9ddb-49f332b1dd9e service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:24.929680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 814/1639] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:24.958651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 826/1640] 10.222.0.88 () {58 vars in 1062 bytes} [Mon Sep 19 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:24.965651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992cc40-2316-4d36-aa95-de34eb444264 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.966176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992cc40-2316-4d36-aa95-de34eb444264 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.966642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992cc40-2316-4d36-aa95-de34eb444264 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:24.968289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 815/1641] 10.222.0.88 () {54 vars in 912 bytes} [Mon Sep 19 11:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:57:24.975908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a3ee3-9a04-4cac-8733-f669e0d3c7e7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:24.976614 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a3ee3-9a04-4cac-8733-f669e0d3c7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:24.977153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a3ee3-9a04-4cac-8733-f669e0d3c7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:25.035006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-364a3ee3-9a04-4cac-8733-f669e0d3c7e7 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:25.036622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-364a3ee3-9a04-4cac-8733-f669e0d3c7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:25.048474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 827/1642] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:25.096980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992cc40-2316-4d36-aa95-de34eb444264 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:25.117881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992cc40-2316-4d36-aa95-de34eb444264 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:25.131228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:25.132866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:25.133250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:25.133603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:25.168110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:25.171112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:25.200906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 816/1643] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:57:25.231206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a3ee3-9a04-4cac-8733-f669e0d3c7e7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:25.250038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a3ee3-9a04-4cac-8733-f669e0d3c7e7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:25.255924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71133c81-20cc-4f3f-937d-c130d01d5741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:25.256551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133c81-20cc-4f3f-937d-c130d01d5741 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:25.256891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133c81-20cc-4f3f-937d-c130d01d5741 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:25.257345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133c81-20cc-4f3f-937d-c130d01d5741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:25.389123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71133c81-20cc-4f3f-937d-c130d01d5741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:25.391679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71133c81-20cc-4f3f-937d-c130d01d5741 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:25.410554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 828/1644] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:28.191343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:28.287248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95284f92-13dc-4842-aed5-3b4684f7fc96 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:28.292925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:28.295021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:28.295319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:28.295593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] PATH_INFO: `/v3/users/ae4ccb010348465e9adf93d156f98fe8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:28.296901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae4ccb010348465e9adf93d156f98fe8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:28.300707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:28.386336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae4ccb010348465e9adf93d156f98fe8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:28.390580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae4ccb010348465e9adf93d156f98fe8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:28.400796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae4ccb010348465e9adf93d156f98fe8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:28.424411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae4ccb010348465e9adf93d156f98fe8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:28.458093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae4ccb010348465e9adf93d156f98fe8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:28.474998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 817/1645] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:28 2022] DELETE /identity/v3/users/ae4ccb010348465e9adf93d156f98fe8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:28.479490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:28.479987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:28.479987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:28.513557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:28.514966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:28.564533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:28.621341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 829/1646] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:28.637389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:28.641513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:28.771938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:28.807798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb568a0-9fb7-4576-8f77-92bfae534815 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:28.812697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:28.814317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:28.819257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:28.819903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:28.841723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:28.846158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:28.869264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 818/1647] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:30.209903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:30.211548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:30.250627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c4db0-340a-404c-81db-91c7c94ae0ef None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:30.269643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:30.270416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:30.270590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:30.270861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] PATH_INFO: `/v3/projects/5c3286ac6e9d49ebaba04fcb5d579855` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:30.273810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c3286ac6e9d49ebaba04fcb5d579855)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:30.275514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:30.447034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c3286ac6e9d49ebaba04fcb5d579855', 'request_id': 'req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:30.463817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c3286ac6e9d49ebaba04fcb5d579855', 'request_id': 'req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:30.482084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c3286ac6e9d49ebaba04fcb5d579855', 'request_id': 'req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:30.529950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] The token cache is being invalidate because project 5c3286ac6e9d49ebaba04fcb5d579855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:30.530482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:30.534054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:30.538471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 830/1648] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:30 2022] DELETE /identity/v3/projects/5c3286ac6e9d49ebaba04fcb5d579855 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:30.542308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:30.554007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc6b4-45f3-4b4b-8528-874662241bf4 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:30.805788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:30.806647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:30.806966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:30.807224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] PATH_INFO: `/v3/users/523e101548424731ada32b29c091afa2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:30.811920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=523e101548424731ada32b29c091afa2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:30.814528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:30.964842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523e101548424731ada32b29c091afa2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:30.966024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523e101548424731ada32b29c091afa2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:30.993326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523e101548424731ada32b29c091afa2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:31.017900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523e101548424731ada32b29c091afa2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:31.059466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341c587-09d7-4e90-a5f8-a81f16c34f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523e101548424731ada32b29c091afa2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:31.080208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 819/1649] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:30 2022] DELETE /identity/v3/users/523e101548424731ada32b29c091afa2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:32.120851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:32.122353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:32.157846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566cdb2-59d7-42ac-941f-bb6ac4fb42ce admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:32.327412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:32.328309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:32.328757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:32.329249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] PATH_INFO: `/v3/projects/b4d7da9c939742299a69303da69e0b66` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:32.333218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4d7da9c939742299a69303da69e0b66)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:32.336388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:32.462769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4d7da9c939742299a69303da69e0b66', 'request_id': 'req-66bfdced-5c92-49ce-a188-540b2d921405'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:32.472850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4d7da9c939742299a69303da69e0b66', 'request_id': 'req-66bfdced-5c92-49ce-a188-540b2d921405'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:32.475889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:32.502053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4d7da9c939742299a69303da69e0b66', 'request_id': 'req-66bfdced-5c92-49ce-a188-540b2d921405'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:32.526751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] The token cache is being invalidate because project b4d7da9c939742299a69303da69e0b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:32.527480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:32.528378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-66bfdced-5c92-49ce-a188-540b2d921405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:32.530615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 831/1650] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:32 2022] DELETE /identity/v3/projects/b4d7da9c939742299a69303da69e0b66 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:32.536514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a929ea-c0df-4a5d-89c3-a53e57a8294b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:32.537013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a929ea-c0df-4a5d-89c3-a53e57a8294b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:32.537102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a929ea-c0df-4a5d-89c3-a53e57a8294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:32.633192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-11a929ea-c0df-4a5d-89c3-a53e57a8294b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:32.635174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-11a929ea-c0df-4a5d-89c3-a53e57a8294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:32.727852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:32.743946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 820/1651] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:32.756328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b1e3e-95ba-4ce8-8911-d807a769fb2e None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:32.759393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b1e3e-95ba-4ce8-8911-d807a769fb2e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:32.760960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b1e3e-95ba-4ce8-8911-d807a769fb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:32.831397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8a5b1e3e-95ba-4ce8-8911-d807a769fb2e None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:32.832695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8a5b1e3e-95ba-4ce8-8911-d807a769fb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:32.862583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 832/1652] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:32.863973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a929ea-c0df-4a5d-89c3-a53e57a8294b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:32.877677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a929ea-c0df-4a5d-89c3-a53e57a8294b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:32.884601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:32.885369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:32.885804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:32.886165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:32.886958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:32.887378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:32.889147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:32.931585 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 821/1653] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:32.937794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b1e3e-95ba-4ce8-8911-d807a769fb2e None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:32.950331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b1e3e-95ba-4ce8-8911-d807a769fb2e None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:32.962924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:32.964115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:32.964550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:32.964976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:32.965811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:32.969326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:33.024279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 833/1654] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:32 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:33.027313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:33.044817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef5bad-791c-41d2-b407-8b6ca230bc3c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:33.186176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:33.188989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:33.189526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:33.190028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:33.191115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:33.193695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:33.239455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 822/1655] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:33 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:33.243688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:33.257568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d8bb-f536-409d-827b-b798e6c67611 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:33.270871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:33.272407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:33.272919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:33.276749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:33.277664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:33.280248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:33.306695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 834/1656] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:33.311478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:33.324261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7711c-d8f2-4a2e-a3a4-094e0792721b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:33.330840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6c0a4-d3db-4cf0-a159-0ad7a68caae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:33.331731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6c0a4-d3db-4cf0-a159-0ad7a68caae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:33.332167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6c0a4-d3db-4cf0-a159-0ad7a68caae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:33.332593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6c0a4-d3db-4cf0-a159-0ad7a68caae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:33.333480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6c0a4-d3db-4cf0-a159-0ad7a68caae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:33.335564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6c0a4-d3db-4cf0-a159-0ad7a68caae6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:33.357963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 823/1657] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:33.363425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:33.377958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7c5da-111a-47ee-849f-32e0f7eb7e17 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:33.389582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:33.390440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:33.390799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:33.391143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] PATH_INFO: `/v3/projects/857a820dc5d24202907006f21080a8d2/users/a42722eb92b3428e9ff435f6540470a8/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:33.417874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] RBAC: Authorizing `identity:create_grant(project_id=857a820dc5d24202907006f21080a8d2, user_id=a42722eb92b3428e9ff435f6540470a8, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:33.421715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:33.443047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 835/1658] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:33 2022] PUT /identity/v3/projects/857a820dc5d24202907006f21080a8d2/users/a42722eb92b3428e9ff435f6540470a8/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:33.455031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e91c0c-a594-4f22-b3ea-3d695b6107c4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:33.455429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e91c0c-a594-4f22-b3ea-3d695b6107c4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:33.455853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e91c0c-a594-4f22-b3ea-3d695b6107c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:33.493213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-27e91c0c-a594-4f22-b3ea-3d695b6107c4 None None] MFA Rules not processed for user `a42722eb92b3428e9ff435f6540470a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:33.495482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-27e91c0c-a594-4f22-b3ea-3d695b6107c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:33.593101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 824/1659] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:33.598045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:33.808445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2df23a-b140-4970-b26b-2c454d2c455b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:33.814741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:33.815749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:33.816826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:33.816826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:33.817700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:33.821972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:33.914172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 836/1660] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:33 2022] POST /identity/v3/projects => generated 381 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:57:33.918490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e91c0c-a594-4f22-b3ea-3d695b6107c4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:33.950716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e91c0c-a594-4f22-b3ea-3d695b6107c4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:34.092429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:34.108066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:34.109265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:34.110016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:34.110856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:34.113200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:34.133321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:34.177218 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 825/1661] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:33 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:57:34.185386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:34.212258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a052e10-729f-4a80-9f8f-5298a440c299 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:34.226592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:34.228180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:34.231024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:34.233214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:34.234658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:34.237273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:34.262359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 837/1662] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:34.272053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:34.301389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a331c-737b-4541-a6eb-6aa634d91431 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:34.310279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:34.311375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:34.311952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:34.312433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] PATH_INFO: `/v3/projects/cb96e7fcbd6047ee822413e89ef19433/users/7db9f910933c4f45bbbd1f46980f43ae/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:34.348518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb96e7fcbd6047ee822413e89ef19433, user_id=7db9f910933c4f45bbbd1f46980f43ae, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:34.351955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:34.375940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 826/1663] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:34 2022] PUT /identity/v3/projects/cb96e7fcbd6047ee822413e89ef19433/users/7db9f910933c4f45bbbd1f46980f43ae/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:57:34.379729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:34.394267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136b8fc-f100-43d2-82b7-7197dbf0b1fd admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:34.560283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:34.583875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519153f0-e93b-4a38-a7c2-06cfb1777ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:34.584698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519153f0-e93b-4a38-a7c2-06cfb1777ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:34.585198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519153f0-e93b-4a38-a7c2-06cfb1777ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:34.585483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519153f0-e93b-4a38-a7c2-06cfb1777ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:34.586180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519153f0-e93b-4a38-a7c2-06cfb1777ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:34.587797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519153f0-e93b-4a38-a7c2-06cfb1777ea6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:34.630363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 838/1664] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:34.631741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:34.669611 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d71d36-2c46-45a0-bcd6-9188bbdf2a87 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:34.684717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:34.685416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:34.685695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:34.685992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] PATH_INFO: `/v3/projects/cb96e7fcbd6047ee822413e89ef19433/users/7db9f910933c4f45bbbd1f46980f43ae/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:34.695978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb96e7fcbd6047ee822413e89ef19433, user_id=7db9f910933c4f45bbbd1f46980f43ae, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:34.698716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:34.741292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 827/1665] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:34 2022] PUT /identity/v3/projects/cb96e7fcbd6047ee822413e89ef19433/users/7db9f910933c4f45bbbd1f46980f43ae/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:34.742839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:34.743131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:34.743413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:34.856363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] MFA Rules not processed for user `7db9f910933c4f45bbbd1f46980f43ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:34.860925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:35.174760 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 839/1666] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:35.242343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:35.242343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:35.248947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:35.250362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:35.406341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71704e93-edfd-40a5-b638-9c02d9737b88 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:35.420540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fa844-23b4-4a4f-9e42-789493bfa583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:35.421444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fa844-23b4-4a4f-9e42-789493bfa583 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:35.421895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fa844-23b4-4a4f-9e42-789493bfa583 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:35.423810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fa844-23b4-4a4f-9e42-789493bfa583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:35.431774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01e6c2-a850-4824-98f7-0bf9c53099d4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:35.444863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fa844-23b4-4a4f-9e42-789493bfa583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:35.445842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:35.446674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:35.448596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:35.449015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] PATH_INFO: `/v3/users/f6e2d26fb24a4ec88cd54088f1150e96` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:35.450150 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fa844-23b4-4a4f-9e42-789493bfa583 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:35.452053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e2d26fb24a4ec88cd54088f1150e96)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:35.456517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:35.481358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 840/1667] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:35.490323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:35.491077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:35.491494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:35.576380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] MFA Rules not processed for user `8fa4b6af47384c7fb26aae3547d02b64`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:35.577612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:35.593258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e2d26fb24a4ec88cd54088f1150e96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:35.594027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e2d26fb24a4ec88cd54088f1150e96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:35.608507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e2d26fb24a4ec88cd54088f1150e96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:35.622567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e2d26fb24a4ec88cd54088f1150e96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:35.654359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c71b6555-3af8-498e-ad1d-f24dcf7024e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e2d26fb24a4ec88cd54088f1150e96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:35.668671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 828/1668] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:35 2022] DELETE /identity/v3/users/f6e2d26fb24a4ec88cd54088f1150e96 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:35.674069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:35.675039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:35.677641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:35.725125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:35.730907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:35.782718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:35.789598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 841/1669] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:35.804922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:35.807782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:35.846855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:35.902779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 829/1670] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:35.913312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:35.914633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:36.017900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f2bbd-8af0-4a86-95c1-cee6eae3ef13 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:36.085088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06787ee0-c851-4d54-a492-db1a3e4e4f6c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:36.096919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:36.097563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:36.097869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:36.098145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:36.124123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:36.130345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:36.172369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 830/1671] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:57:36.174206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:36.176278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:36.176278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:36.176484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:36.176749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:36.178583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:36.356891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:36.367061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:36.389811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e13fde-bd35-46a7-b27c-9dbc9d135bae service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:36.403662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db9f910933c4f45bbbd1f46980f43ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb96e7fcbd6047ee822413e89ef19433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:36.405006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:36.405301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:36.405581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:36.406299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:36.406606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:36.408512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 842/1672] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:57:36.411439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:36.442039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 831/1673] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:36.443256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:36.475520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c85cd5-11f6-433a-a2db-98f9e6a3f1a5 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:36.499349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db9f910933c4f45bbbd1f46980f43ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb96e7fcbd6047ee822413e89ef19433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:36.500014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:36.500334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:36.500614 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:36.504681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:36.504681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:36.589605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 843/1674] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:36 2022] POST /identity/v3/projects => generated 353 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:57:36.594090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:36.608873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414c87e-4184-4fb4-9549-075bcf88b18a tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:36.759447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db9f910933c4f45bbbd1f46980f43ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb96e7fcbd6047ee822413e89ef19433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:36.766460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:36.767003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:36.767733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:36.769619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:36.772528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:36.776723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:36.815874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:36.834888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 832/1675] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:36.879365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:36.896551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49ead7-4974-477e-8f7a-5786de24cb8f tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:36.903979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db9f910933c4f45bbbd1f46980f43ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb96e7fcbd6047ee822413e89ef19433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:36.905178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:36.905917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:36.906273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] PATH_INFO: `/v3/projects/1070986ec6804a12817ac4bb7c981422` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:36.908605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1070986ec6804a12817ac4bb7c981422)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:36.911442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:36.933048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:36.935262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:36.946846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:37.005798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1070986ec6804a12817ac4bb7c981422', 'request_id': 'req-957083f3-dece-4486-acf5-5096b3d901f0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.021083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1070986ec6804a12817ac4bb7c981422', 'request_id': 'req-957083f3-dece-4486-acf5-5096b3d901f0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.033528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1070986ec6804a12817ac4bb7c981422', 'request_id': 'req-957083f3-dece-4486-acf5-5096b3d901f0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.055111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd3944-97f9-4811-8d5b-f69396d9992b tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:37.060519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] The token cache is being invalidate because project 1070986ec6804a12817ac4bb7c981422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:37.060813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.062821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.075761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 844/1676] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:36 2022] DELETE /identity/v3/projects/1070986ec6804a12817ac4bb7c981422 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:37.233647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:37.236464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:37.238123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:37.238566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] PATH_INFO: `/v3/projects/02d3b2d4d255495f91e42bf4c39a41a7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:37.242428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d3b2d4d255495f91e42bf4c39a41a7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:37.244329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:57:37.244329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 19 11:57:37.244329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: Sep 19 11:57:37.245616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:37.354584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d3b2d4d255495f91e42bf4c39a41a7', 'request_id': 'req-38828653-7449-4d38-bc40-46b9eeb4edaa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.384125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d3b2d4d255495f91e42bf4c39a41a7', 'request_id': 'req-38828653-7449-4d38-bc40-46b9eeb4edaa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.400322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d3b2d4d255495f91e42bf4c39a41a7', 'request_id': 'req-38828653-7449-4d38-bc40-46b9eeb4edaa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.424119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] The token cache is being invalidate because project 02d3b2d4d255495f91e42bf4c39a41a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:37.426013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.428793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38828653-7449-4d38-bc40-46b9eeb4edaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.439037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:37.441049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:37.449186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 833/1677] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:36 2022] DELETE /identity/v3/projects/02d3b2d4d255495f91e42bf4c39a41a7 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:37.451472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:37.451975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:37.452396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:37.474600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957083f3-dece-4486-acf5-5096b3d901f0 tempest-VolumeQuotaClassesTest-1244179785 tempest-VolumeQuotaClassesTest-1244179785-project-admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:37.530945 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:37.533298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:37.597407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:37.700909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:37.701489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:37.701787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:37.703386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] PATH_INFO: `/v3/users/a42722eb92b3428e9ff435f6540470a8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:37.705592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42722eb92b3428e9ff435f6540470a8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:37.706722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: WARNING py.warnings [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:57:37.706722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: warnings.warn(msg) Sep 19 11:57:37.706722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: Sep 19 11:57:37.709520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:37.719535 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 834/1678] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:37.719535 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:37.719535 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:37.740096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9466db-35bf-4dd6-84e5-e247b63e749c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:37.760007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:37.760771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:37.761081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:37.761300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] PATH_INFO: `/v3/users/4ece529671e94fc6a9beba42631d2bd3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:37.763119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ece529671e94fc6a9beba42631d2bd3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:37.768630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:37.835176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42722eb92b3428e9ff435f6540470a8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.858421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42722eb92b3428e9ff435f6540470a8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.859322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42722eb92b3428e9ff435f6540470a8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.875830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ece529671e94fc6a9beba42631d2bd3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.877154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ece529671e94fc6a9beba42631d2bd3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.884392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42722eb92b3428e9ff435f6540470a8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.895034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ece529671e94fc6a9beba42631d2bd3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.913878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-73fa4b60-d037-45cf-b44c-f877d90c89c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42722eb92b3428e9ff435f6540470a8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.919289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ece529671e94fc6a9beba42631d2bd3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.928826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 845/1679] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:37 2022] DELETE /identity/v3/users/a42722eb92b3428e9ff435f6540470a8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:57:37.930862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43e17c-6516-4d44-88d5-db895ce80806 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:37.932785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43e17c-6516-4d44-88d5-db895ce80806 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:37.934235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43e17c-6516-4d44-88d5-db895ce80806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:37.948713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09f02f69-4972-416f-83b9-d077ddb71aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ece529671e94fc6a9beba42631d2bd3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:37.965871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0b43e17c-6516-4d44-88d5-db895ce80806 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:37.970453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 835/1680] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:37 2022] DELETE /identity/v3/users/4ece529671e94fc6a9beba42631d2bd3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:37.973025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:37.973708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:37.974533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:37.974943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0b43e17c-6516-4d44-88d5-db895ce80806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:38.005633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:38.008989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:38.012685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:38.070156 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 846/1681] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:38.075153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6577e0-f909-4945-bb9a-fd570091e801 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:38.075444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6577e0-f909-4945-bb9a-fd570091e801 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:38.075724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6577e0-f909-4945-bb9a-fd570091e801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:38.151245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-cd6577e0-f909-4945-bb9a-fd570091e801 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:38.152338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cd6577e0-f909-4945-bb9a-fd570091e801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:38.190024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 847/1682] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:38.190925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6577e0-f909-4945-bb9a-fd570091e801 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:38.217981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6577e0-f909-4945-bb9a-fd570091e801 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:38.240933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:38.241548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 836/1683] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:38.242720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:38.243670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:38.245548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:38.246685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:38.247498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:38.248611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:38.248860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:38.251836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:38.287785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 848/1684] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:38.289730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:38.295741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:38.440280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:38.457046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc762e-f7f2-43d9-b5f9-ac55c669cca5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:38.466778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:38.470735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:38.471801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:38.473662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:38.508960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:38.510728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:38.522624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b745a82-9ce0-4870-bc51-db98ea28b134 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:38.536304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:38.540995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:38.541740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:38.542370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:38.554459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 837/1685] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:38.558166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:38.567895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:38.569439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:38.580690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c792e2-2385-4855-982c-e8599f19a63f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:38.595518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31b0cc-d706-4a37-877e-c7f989c92b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:38.603122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31b0cc-d706-4a37-877e-c7f989c92b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:38.606503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 849/1686] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:38.610021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31b0cc-d706-4a37-877e-c7f989c92b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:38.610717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31b0cc-d706-4a37-877e-c7f989c92b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:38.611702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31b0cc-d706-4a37-877e-c7f989c92b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:38.615882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31b0cc-d706-4a37-877e-c7f989c92b65 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:38.728569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 838/1687] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:38 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:38.732691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:38.751930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814f45e-9b0e-4375-91fd-96f5165878ec service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:38.905397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bfe04-2646-4847-a888-10c45b5aa4fc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:38.905920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bfe04-2646-4847-a888-10c45b5aa4fc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:38.906536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bfe04-2646-4847-a888-10c45b5aa4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:39.018965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:39.020081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:39.022568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:39.024392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:39.027432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:39.031097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:39.078475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f19bfe04-2646-4847-a888-10c45b5aa4fc None None] MFA Rules not processed for user `11af9e7e3fed4bea9187dee0ebf27551`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:39.087798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f19bfe04-2646-4847-a888-10c45b5aa4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:39.095462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 850/1688] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:38 2022] POST /identity/v3/users => generated 291 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:39.099064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:39.113506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4752e-6b45-4a49-b096-d9942fc67811 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:39.131664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:39.133621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:39.139471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:39.141933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:39.144084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:39.152538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:39.205847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 851/1689] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:39.208117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:39.237852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81337802-b070-45e0-9be2-71abdac70251 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:39.267028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:39.267756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:39.268226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:39.268585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:39.269311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:39.277452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:39.302406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 839/1690] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:39.302506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 852/1691] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:39.306250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:39.306627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:39.307767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:39.311741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:39.356427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb84c3-b3e5-4f84-a6f1-5370ffe69dd2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:39.364241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] MFA Rules not processed for user `11af9e7e3fed4bea9187dee0ebf27551`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:39.367034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:39.370449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ba4b5-0c19-4f69-8917-4bbbc9ee7f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:39.371327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ba4b5-0c19-4f69-8917-4bbbc9ee7f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:39.371858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ba4b5-0c19-4f69-8917-4bbbc9ee7f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:39.372237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ba4b5-0c19-4f69-8917-4bbbc9ee7f0e admin admin] PATH_INFO: `/v3/projects/0b356e0c6337455593a3441e44d93790/users/cfe2be353f0c409c82e1e269e25cf34c/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:39.395614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 840/1692] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:39.409629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:39.413595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:39.435797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ba4b5-0c19-4f69-8917-4bbbc9ee7f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b356e0c6337455593a3441e44d93790, user_id=cfe2be353f0c409c82e1e269e25cf34c, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:39.439707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ba4b5-0c19-4f69-8917-4bbbc9ee7f0e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:39.449917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:39.494822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 853/1693] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:39 2022] PUT /identity/v3/projects/0b356e0c6337455593a3441e44d93790/users/cfe2be353f0c409c82e1e269e25cf34c/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:39.508367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6803-dbe8-4426-8174-266618ec85ba None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:39.508803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6803-dbe8-4426-8174-266618ec85ba None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:39.509174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6803-dbe8-4426-8174-266618ec85ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:39.593024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-16fa6803-dbe8-4426-8174-266618ec85ba None None] MFA Rules not processed for user `cfe2be353f0c409c82e1e269e25cf34c`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:39.597715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-16fa6803-dbe8-4426-8174-266618ec85ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:39.688424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13cf6a-3eb3-4f1f-b81a-d1036ed21874 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:39.779546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 854/1694] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:39.780993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6803-dbe8-4426-8174-266618ec85ba None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:39.831830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:39.911131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:39.914923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:39.916350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:39.917328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:40.038310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6803-dbe8-4426-8174-266618ec85ba None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:40.061643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:40.062356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:40.062635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:40.062909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:40.063611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:40.065790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:40.142137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:40.143784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:40.160557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 855/1695] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:39 2022] POST /identity/v3/projects => generated 379 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:40.163422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:40.172327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 841/1696] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:40.394310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da96d13-ea0d-4bb3-af9d-f65c86d2d5e9 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:40.416353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:40.418928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:40.420364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:40.420811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:40.426288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:40.428310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:40.491752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:40.518033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:40.523187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 856/1697] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:40 2022] POST /identity/v3/users => generated 296 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:40.540806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfca9fe-f28f-4858-8b1a-49557ae57d70 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:40.589789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:40.589789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:40.589789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:40.589789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:40.590205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:40.604015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:40.658128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:40.664528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 842/1698] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:40.670264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2601e66-d786-4ebd-9ee4-f473e46bf4ee admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:40.699210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:40.700647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:40.702000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:40.703565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] PATH_INFO: `/v3/projects/405638a816464e0ca2a0808c235950b7/users/bc0993c3236744208a86ab5e1796ec2d/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:40.756843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405638a816464e0ca2a0808c235950b7, user_id=bc0993c3236744208a86ab5e1796ec2d, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:40.760561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:40.796738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 857/1699] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:40 2022] PUT /identity/v3/projects/405638a816464e0ca2a0808c235950b7/users/bc0993c3236744208a86ab5e1796ec2d/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:57:40.801204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:40.821313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d15499-51d1-4600-94a4-5ab3d375a207 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:40.975509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f19041-9ae7-408a-b957-356eb4cea44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:40.978216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19041-9ae7-408a-b957-356eb4cea44a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:40.979762 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19041-9ae7-408a-b957-356eb4cea44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:40.980074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19041-9ae7-408a-b957-356eb4cea44a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:40.980855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f19041-9ae7-408a-b957-356eb4cea44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:40.987070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f19041-9ae7-408a-b957-356eb4cea44a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:41.018953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 843/1700] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:41.020373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:41.040996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08b8e-d928-44d0-a5e3-43dffd165a29 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:41.054644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:41.056726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:41.058385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:41.058888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] PATH_INFO: `/v3/projects/405638a816464e0ca2a0808c235950b7/users/bc0993c3236744208a86ab5e1796ec2d/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:41.067653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405638a816464e0ca2a0808c235950b7, user_id=bc0993c3236744208a86ab5e1796ec2d, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:41.083274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:41.104024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 858/1701] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:41 2022] PUT /identity/v3/projects/405638a816464e0ca2a0808c235950b7/users/bc0993c3236744208a86ab5e1796ec2d/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:41.114419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:41.114717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:41.114978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:41.203309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] MFA Rules not processed for user `bc0993c3236744208a86ab5e1796ec2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:41.212990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:41.312291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:41.316615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:41.458999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:41.472193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 844/1702] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:41.487977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db169a5-d4c1-4819-99a3-a3468a4a5354 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:41.494905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:41.496621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:41.502702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:41.503672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:41.504680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:41.505108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] PATH_INFO: `/v3/projects/572b24f6b830413ca03007b0eb1c7695` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:41.507055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=572b24f6b830413ca03007b0eb1c7695)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:41.519912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:41.648079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572b24f6b830413ca03007b0eb1c7695', 'request_id': 'req-f45f42bb-ff58-45f3-922d-fbc034a89686'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:41.663740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84887a1b-56e6-41c5-8d18-c72e5ffb1e07 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:41.667984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572b24f6b830413ca03007b0eb1c7695', 'request_id': 'req-f45f42bb-ff58-45f3-922d-fbc034a89686'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:41.682735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572b24f6b830413ca03007b0eb1c7695', 'request_id': 'req-f45f42bb-ff58-45f3-922d-fbc034a89686'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:41.728721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] The token cache is being invalidate because project 572b24f6b830413ca03007b0eb1c7695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:41.734855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:41.738569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:41.745138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 859/1703] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:41 2022] DELETE /identity/v3/projects/572b24f6b830413ca03007b0eb1c7695 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:57:41.753137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:41.761926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:41.845989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cce270-353f-4bac-b8ce-84ed49b33365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:41.846686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce270-353f-4bac-b8ce-84ed49b33365 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:41.847208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce270-353f-4bac-b8ce-84ed49b33365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:41.847531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce270-353f-4bac-b8ce-84ed49b33365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:41.855424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-24cce270-353f-4bac-b8ce-84ed49b33365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:41.975872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:42.040257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f42bb-ff58-45f3-922d-fbc034a89686 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:42.049009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cce270-353f-4bac-b8ce-84ed49b33365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:42.055471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cce270-353f-4bac-b8ce-84ed49b33365 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:42.059679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:42.060704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:42.061082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:42.061934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] PATH_INFO: `/v3/projects/857a820dc5d24202907006f21080a8d2` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:42.064667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=857a820dc5d24202907006f21080a8d2)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:42.075587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:42.101423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 845/1704] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:42.102910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba67a36-f162-47d0-bb74-275cff6eaaf1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:42.103201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba67a36-f162-47d0-bb74-275cff6eaaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:42.107533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba67a36-f162-47d0-bb74-275cff6eaaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:42.161269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eba67a36-f162-47d0-bb74-275cff6eaaf1 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:42.167049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eba67a36-f162-47d0-bb74-275cff6eaaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:42.192709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 846/1705] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:42.196515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fe427-1282-475a-b434-03b28e892aef None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:42.196775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fe427-1282-475a-b434-03b28e892aef None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:42.197082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fe427-1282-475a-b434-03b28e892aef None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:42.201200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 847/1706] 10.222.0.88 () {54 vars in 912 bytes} [Mon Sep 19 11:57:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:57:42.208823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:42.209939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:42.211121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:42.222328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857a820dc5d24202907006f21080a8d2', 'request_id': 'req-4f863f0c-dc5c-4055-b246-d64821d72e9c'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.239651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857a820dc5d24202907006f21080a8d2', 'request_id': 'req-4f863f0c-dc5c-4055-b246-d64821d72e9c'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.254314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857a820dc5d24202907006f21080a8d2', 'request_id': 'req-4f863f0c-dc5c-4055-b246-d64821d72e9c'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.264346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:42.267651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:42.282955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] The token cache is being invalidate because project 857a820dc5d24202907006f21080a8d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:42.283719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.285198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4f863f0c-dc5c-4055-b246-d64821d72e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.287938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 860/1707] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:41 2022] DELETE /identity/v3/projects/857a820dc5d24202907006f21080a8d2 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:42.291627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:42.292181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:42.293625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:42.443530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:42.448427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] MFA Rules not processed for user `dee035c97cb6447382365faf6b6aca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:42.450514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:42.478987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 848/1708] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:42.482679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:42.492522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:42.528663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621b8b9-f504-45ac-ba2f-1b32598d6b50 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:42.548392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:42.550699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:42.551186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:42.554061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] PATH_INFO: `/v3/users/7db9f910933c4f45bbbd1f46980f43ae` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:42.556889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db9f910933c4f45bbbd1f46980f43ae)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:42.559642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:42.639978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 861/1709] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:57:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:42.640914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:42.642398 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:42.704339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db9f910933c4f45bbbd1f46980f43ae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.706150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db9f910933c4f45bbbd1f46980f43ae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.727249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db9f910933c4f45bbbd1f46980f43ae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.752953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db9f910933c4f45bbbd1f46980f43ae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.775664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9cbb7-050a-427f-8efa-5ba44d9891b3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:42.783597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db9f910933c4f45bbbd1f46980f43ae'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:42.787514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8252a9-beb9-4184-b619-bad6fb4d5478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:42.788424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8252a9-beb9-4184-b619-bad6fb4d5478 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:42.788830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8252a9-beb9-4184-b619-bad6fb4d5478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:42.789202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8252a9-beb9-4184-b619-bad6fb4d5478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:42.794566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-dd8252a9-beb9-4184-b619-bad6fb4d5478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:42.800826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 849/1710] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:42 2022] DELETE /identity/v3/users/7db9f910933c4f45bbbd1f46980f43ae => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:42.802555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:42.806323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:42.935578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:42.997131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8252a9-beb9-4184-b619-bad6fb4d5478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:43.001503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8252a9-beb9-4184-b619-bad6fb4d5478 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:43.003402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472532e-12b3-4f05-838e-c045f769c8d9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:43.016884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:43.017529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.020674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 862/1711] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:43.022342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af90ac-e23a-4e8f-84f1-8a7eab72d5e5 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.022870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af90ac-e23a-4e8f-84f1-8a7eab72d5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.023588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af90ac-e23a-4e8f-84f1-8a7eab72d5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.032771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.033253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.037144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:43.037449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:43.062442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:43.083517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a5af90ac-e23a-4e8f-84f1-8a7eab72d5e5 None None] MFA Rules not processed for user `1015b24d41f446069b0e0884b92a1772`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:43.088249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a5af90ac-e23a-4e8f-84f1-8a7eab72d5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:43.123883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 850/1712] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:43.128878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:43.136587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:43.298453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 863/1713] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:43.299939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b3e5-5d11-42c8-8def-3c49ee880bff None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.300229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b3e5-5d11-42c8-8def-3c49ee880bff None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.300501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b3e5-5d11-42c8-8def-3c49ee880bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.330519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5ab0b3e5-5d11-42c8-8def-3c49ee880bff None None] MFA Rules not processed for user `8fa4b6af47384c7fb26aae3547d02b64`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:43.331732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5ab0b3e5-5d11-42c8-8def-3c49ee880bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:43.340837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab06c-797e-4a77-b592-49fc2b04967e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:43.346477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:43.347321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.347851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.348331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.367467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:43.392510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:43.393713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:43.420486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 851/1714] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:43.422421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:43.439913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814caa-ed48-4e6f-8544-9e1ca30e15f1 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:43.448187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 864/1715] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:43.450385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc42b-d9f3-4b24-ab29-5a02c2930c64 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.451160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f6710-bd8b-42fb-b263-d74dc01a4ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:43.452336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc42b-d9f3-4b24-ab29-5a02c2930c64 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.452720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc42b-d9f3-4b24-ab29-5a02c2930c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.453572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f6710-bd8b-42fb-b263-d74dc01a4ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.454028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f6710-bd8b-42fb-b263-d74dc01a4ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.455011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f6710-bd8b-42fb-b263-d74dc01a4ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.456359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f6710-bd8b-42fb-b263-d74dc01a4ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:43.460323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f6710-bd8b-42fb-b263-d74dc01a4ef3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:43.471750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:43.521545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 852/1716] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:43 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:43.524098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.524379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.526738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-49bdc42b-d9f3-4b24-ab29-5a02c2930c64 None None] MFA Rules not processed for user `766511b86ad643c099998fc4e1638f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:43.527999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.529259 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-49bdc42b-d9f3-4b24-ab29-5a02c2930c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:43.597727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] MFA Rules not processed for user `1015b24d41f446069b0e0884b92a1772`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:43.603113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:43.728736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:43.761992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 865/1717] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:43.762999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc42b-d9f3-4b24-ab29-5a02c2930c64 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:43.796886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 853/1718] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:43.798948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:43.802659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:43.911121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc42b-d9f3-4b24-ab29-5a02c2930c64 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:43.918883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:43.919908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.920489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.921089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.921857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:43.929982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:43.954298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f2f3-552d-40cb-8a9c-36c5908679a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:43.964650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:43.966371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:43.966872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:43.967460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] PATH_INFO: `/v3/projects/cb96e7fcbd6047ee822413e89ef19433` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:43.969884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb96e7fcbd6047ee822413e89ef19433)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:43.973991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:43.980954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 866/1719] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:43 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:43.986653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:44.010037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208e547-a188-400f-bca0-9355ae5b3d4c admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:44.031764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:44.034335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.036917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.037513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.038477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:44.044129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:44.067103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb96e7fcbd6047ee822413e89ef19433', 'request_id': 'req-cabb5848-46e1-4090-ab21-eb45da6276ed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:44.074098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 867/1720] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:44.079897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:44.085410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb96e7fcbd6047ee822413e89ef19433', 'request_id': 'req-cabb5848-46e1-4090-ab21-eb45da6276ed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:44.104137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb96e7fcbd6047ee822413e89ef19433', 'request_id': 'req-cabb5848-46e1-4090-ab21-eb45da6276ed'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:44.107067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc73cc-b1cb-4412-890f-18dffbc0688b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:44.144793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] The token cache is being invalidate because project cb96e7fcbd6047ee822413e89ef19433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:44.145802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:44.150193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cabb5848-46e1-4090-ab21-eb45da6276ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:44.158577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 854/1721] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:43 2022] DELETE /identity/v3/projects/cb96e7fcbd6047ee822413e89ef19433 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:44.166101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadfae3-541e-4f08-b1a4-a25a1a70cb52 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.166696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadfae3-541e-4f08-b1a4-a25a1a70cb52 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.167182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadfae3-541e-4f08-b1a4-a25a1a70cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.196313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-beadfae3-541e-4f08-b1a4-a25a1a70cb52 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:44.198408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-beadfae3-541e-4f08-b1a4-a25a1a70cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:44.222338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:44.307587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:44.308216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.308547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.309968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.311656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:44.317693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:44.350587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 855/1722] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:57:44.361745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 868/1723] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:44.364236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:44.367469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:44.371910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.372394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.373145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.417829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661fe23-8124-4ed9-8e38-9e025f077cc2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:44.439002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:44.439748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.440047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.440286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] PATH_INFO: `/v3/projects/7869613d45064c3e9b07fd3d41183637/users/10fba9b80ee2408aa916add7397b3348/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.450123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:44.456203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:44.514149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 856/1724] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:44.514728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:44.521299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:44.524016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7869613d45064c3e9b07fd3d41183637, user_id=10fba9b80ee2408aa916add7397b3348, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:44.535560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:44.590812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 869/1725] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:44 2022] PUT /identity/v3/projects/7869613d45064c3e9b07fd3d41183637/users/10fba9b80ee2408aa916add7397b3348/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:44.595778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:44.709959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:44.733067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7a2a0-902a-4e92-bf9b-20f9c083448d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:44.739806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2fece-3cad-4df9-a111-844b254ef6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:44.740638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2fece-3cad-4df9-a111-844b254ef6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.741249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2fece-3cad-4df9-a111-844b254ef6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.741655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2fece-3cad-4df9-a111-844b254ef6b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.742579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2fece-3cad-4df9-a111-844b254ef6b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:44.742986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2fece-3cad-4df9-a111-844b254ef6b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:44.745415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2fece-3cad-4df9-a111-844b254ef6b4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:44.767049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e275bd-8742-4879-bd2d-07ac048c35ee None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:44.783878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 870/1726] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:44.785001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf5bc-5524-4f74-86ce-639be7d75537 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.785405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf5bc-5524-4f74-86ce-639be7d75537 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.787462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf5bc-5524-4f74-86ce-639be7d75537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.865467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4acbf5bc-5524-4f74-86ce-639be7d75537 None None] MFA Rules not processed for user `10fba9b80ee2408aa916add7397b3348`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:44.868496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4acbf5bc-5524-4f74-86ce-639be7d75537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:44.939785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:44.942216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:44.943230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:44.943588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:44.950861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:45.003245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 871/1727] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:45.005373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a9cf-e093-4f44-aa44-3d9bc2e987c0 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.007262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a9cf-e093-4f44-aa44-3d9bc2e987c0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.010202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a9cf-e093-4f44-aa44-3d9bc2e987c0 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.024620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 872/1728] 10.222.0.88 () {54 vars in 912 bytes} [Mon Sep 19 11:57:45 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:57:45.025534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a9cf-e093-4f44-aa44-3d9bc2e987c0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:45.028056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8162a9cf-e093-4f44-aa44-3d9bc2e987c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:45.079729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:45.105163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:45.110023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:45.148270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 857/1729] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:45.157515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a9cf-e093-4f44-aa44-3d9bc2e987c0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:45.162089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:45.165529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f172d-befa-486f-ae04-e67926b6c9a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:45.166461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f172d-befa-486f-ae04-e67926b6c9a8 service placement] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.167880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f172d-befa-486f-ae04-e67926b6c9a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.171321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f172d-befa-486f-ae04-e67926b6c9a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.179234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8e0f172d-befa-486f-ae04-e67926b6c9a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:45.205874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f172d-befa-486f-ae04-e67926b6c9a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:45.207837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f172d-befa-486f-ae04-e67926b6c9a8 service placement] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:45.210520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5baacc1-0d62-47dd-b98d-cbdc688c9ec3 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:45.218424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:45.219192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.220008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.220386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.220916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:45.222861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:45.225860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 873/1730] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:45.231219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668d38-acc6-405e-ac32-75c8e48489b9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.232435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668d38-acc6-405e-ac32-75c8e48489b9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.233002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668d38-acc6-405e-ac32-75c8e48489b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.296831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b2668d38-acc6-405e-ac32-75c8e48489b9 None None] MFA Rules not processed for user `dee035c97cb6447382365faf6b6aca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:45.300982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b2668d38-acc6-405e-ac32-75c8e48489b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:45.349637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 858/1731] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:45 2022] POST /identity/v3/projects => generated 393 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:45.349637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:45.486876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 874/1732] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:57:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:45.489010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2668d38-acc6-405e-ac32-75c8e48489b9 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:45.503044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32473-8067-43d5-a11f-72d54a446a04 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:45.510639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:45.511452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.511807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.512177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.514659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:45.528762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2668d38-acc6-405e-ac32-75c8e48489b9 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:45.539744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:45.605935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:45.608136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:45.633127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 859/1733] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:45.636032 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:45.641847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:45.740975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:45.743186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.747393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.750299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.754140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:45.760640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:45.838944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee0d2f-aac5-4235-950f-ecc37ba37685 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:45.843658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 875/1734] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:45 2022] POST /identity/v3/users => generated 297 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:45.846290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:45.847964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:45.848289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.848575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.848952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.866807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:45.872182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:45.883038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a596-f729-4392-8fc1-65bde8d72450 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:45.898168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 860/1735] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:45.901095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:45.901734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.902043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.902337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.904323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:45.909114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:45.953875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 876/1736] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:45.958896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:45.989219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c04d3-f9bc-42dc-9996-164890d1491f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:45.995439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fa7bd-09f5-49ca-803d-790debce8417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:45.996112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fa7bd-09f5-49ca-803d-790debce8417 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:45.997057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fa7bd-09f5-49ca-803d-790debce8417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:45.997555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fa7bd-09f5-49ca-803d-790debce8417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:45.998174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fa7bd-09f5-49ca-803d-790debce8417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:46.000406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fa7bd-09f5-49ca-803d-790debce8417 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:46.002887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:46.041488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 861/1737] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:46.042918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:46.055874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8dabd2-7fae-435c-aaf8-4c0d38c46671 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:46.063179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:46.064294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:46.064830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:46.065308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] PATH_INFO: `/v3/projects/075fcaf37ea74f2f987976d9ee2b8d8c/users/f90ac5451757467999b63c178e135db1/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:46.085297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=075fcaf37ea74f2f987976d9ee2b8d8c, user_id=f90ac5451757467999b63c178e135db1, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:46.090413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:46.118377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 877/1738] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:46 2022] PUT /identity/v3/projects/075fcaf37ea74f2f987976d9ee2b8d8c/users/f90ac5451757467999b63c178e135db1/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:46.128322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275902d-5ccd-463b-9394-6ee196346fcc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:46.129259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275902d-5ccd-463b-9394-6ee196346fcc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:46.130578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275902d-5ccd-463b-9394-6ee196346fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:46.173935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5275902d-5ccd-463b-9394-6ee196346fcc None None] MFA Rules not processed for user `f90ac5451757467999b63c178e135db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:46.175401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5275902d-5ccd-463b-9394-6ee196346fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:46.279298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 862/1739] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:46.283631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:46.438205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2b41c-8390-4868-ba56-2dece9410ac4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:46.452552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:46.453342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:46.454494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:46.454912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:46.455648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:46.457910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:46.523497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 878/1740] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:46 2022] POST /identity/v3/projects => generated 393 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:46.529130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275902d-5ccd-463b-9394-6ee196346fcc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:46.559825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275902d-5ccd-463b-9394-6ee196346fcc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:46.785265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:46.786088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:46.786953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:46.788350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:46.789920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:46.796628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:46.841058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 863/1741] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:46 2022] POST /identity/v3/users => generated 303 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:46.847848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:46.871832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861469c3-d695-4b1a-a984-5dafdfd78375 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:46.897999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:46.901991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:46.902464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:46.902879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:46.908187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:46.912651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:46.962199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:46.964914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 879/1742] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:46.988662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7f6c2-5339-4bf9-9dcb-4820b5443bc5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:47.042521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:47.043215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:47.043499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:47.043776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] PATH_INFO: `/v3/projects/4b7de59eda3848cf9ce1053f585327cb/users/b16445c9b7e9499c95029d3ff27efc1a/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:47.095836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7de59eda3848cf9ce1053f585327cb, user_id=b16445c9b7e9499c95029d3ff27efc1a, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:47.098889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:47.147254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 864/1743] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:46 2022] PUT /identity/v3/projects/4b7de59eda3848cf9ce1053f585327cb/users/b16445c9b7e9499c95029d3ff27efc1a/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:47.151761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:47.201578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ab35b-84cf-4116-8ca6-469f8391cc65 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:47.288213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:47.327192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:47.395776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:47.435206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:47.438230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:47.441206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:47.442837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:47.444673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:47.445199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e719fc-8624-4923-beb1-b7b24d0890d6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:47.450554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:47.460859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0c63d-b7f3-43de-bc74-c21cebf231be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:47.461991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c63d-b7f3-43de-bc74-c21cebf231be service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:47.462698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c63d-b7f3-43de-bc74-c21cebf231be service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:47.463331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c63d-b7f3-43de-bc74-c21cebf231be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:47.486292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 880/1744] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:47.490785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:47.508134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763e31-7804-49fd-a7c1-95599edd9f29 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:47.546689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef136cd-eaca-42b7-abed-6f6011650559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:47.547288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef136cd-eaca-42b7-abed-6f6011650559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:47.547717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef136cd-eaca-42b7-abed-6f6011650559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:47.548658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef136cd-eaca-42b7-abed-6f6011650559 admin admin] PATH_INFO: `/v3/projects/4b7de59eda3848cf9ce1053f585327cb/users/b16445c9b7e9499c95029d3ff27efc1a/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:47.560093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef136cd-eaca-42b7-abed-6f6011650559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7de59eda3848cf9ce1053f585327cb, user_id=b16445c9b7e9499c95029d3ff27efc1a, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:47.570973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef136cd-eaca-42b7-abed-6f6011650559 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:47.619473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 881/1745] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:47 2022] PUT /identity/v3/projects/4b7de59eda3848cf9ce1053f585327cb/users/b16445c9b7e9499c95029d3ff27efc1a/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:47.620200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:47.620486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:47.623861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:47.681066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] MFA Rules not processed for user `11af9e7e3fed4bea9187dee0ebf27551`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:47.682922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:47.690423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0c63d-b7f3-43de-bc74-c21cebf231be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:47.692057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0c63d-b7f3-43de-bc74-c21cebf231be service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:47.718248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 865/1746] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:57:47.720342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:47.720543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:47.725961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:47.834205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 882/1747] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:47.834446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] MFA Rules not processed for user `b16445c9b7e9499c95029d3ff27efc1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:47.835642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:48.018298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 866/1748] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:48.030039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:48.031712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:48.135733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79749-1ba6-4825-ba3d-75469d0e3968 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:48.150780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:48.161137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:48.161137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:48.161137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:48.198189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:48.200099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:48.250099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 883/1749] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:48 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:48.348711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:48.350826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:48.387872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e57410-d48c-43c2-b62b-745f3bb261ef None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:48.403714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ded34-ced7-4bb8-95eb-be6fd968e79b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:48.404511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ded34-ced7-4bb8-95eb-be6fd968e79b service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:48.404860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ded34-ced7-4bb8-95eb-be6fd968e79b service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:48.405220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ded34-ced7-4bb8-95eb-be6fd968e79b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:48.551034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ded34-ced7-4bb8-95eb-be6fd968e79b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:48.552895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ded34-ced7-4bb8-95eb-be6fd968e79b service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:48.572416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 867/1750] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:48.728974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:48.956128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01585f69-0144-4d95-b373-96cb3a6f6eca service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:48.968444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:48.969970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:48.971237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:48.971716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] PATH_INFO: `/v3/users/10fba9b80ee2408aa916add7397b3348` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:48.973802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10fba9b80ee2408aa916add7397b3348)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:48.976899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:48.991914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:49.046424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10fba9b80ee2408aa916add7397b3348'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:49.060640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10fba9b80ee2408aa916add7397b3348'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:49.061518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10fba9b80ee2408aa916add7397b3348'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:49.086799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10fba9b80ee2408aa916add7397b3348'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:49.193885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10fba9b80ee2408aa916add7397b3348'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:49.215663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 884/1751] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:48 2022] DELETE /identity/v3/users/10fba9b80ee2408aa916add7397b3348 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:49.221094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:49.221372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:49.221647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:49.247998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:49.250741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:49.258536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:49.364167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 868/1752] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:49.378021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:49.380044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:49.496457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51573315-e469-45ae-b6f8-8dfb447215c5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:49.514646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311ab2-e26d-4a2c-b19a-5b32c40a32c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:49.514646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311ab2-e26d-4a2c-b19a-5b32c40a32c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:49.514646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311ab2-e26d-4a2c-b19a-5b32c40a32c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:49.514646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311ab2-e26d-4a2c-b19a-5b32c40a32c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:49.520682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311ab2-e26d-4a2c-b19a-5b32c40a32c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:49.520970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311ab2-e26d-4a2c-b19a-5b32c40a32c9 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:49.538950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 885/1753] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:49.943586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:49.945114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:49.972283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aa2d1-1537-480a-9028-a43f2572c1fd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:49.991312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:49.992075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:49.992365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:49.992640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] PATH_INFO: `/v3/projects/7869613d45064c3e9b07fd3d41183637` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:49.995716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7869613d45064c3e9b07fd3d41183637)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:50.001635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:50.212899 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7869613d45064c3e9b07fd3d41183637', 'request_id': 'req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:50.246668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8363d9-1bb8-469d-8e26-07da5db475dc None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:50.247758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8363d9-1bb8-469d-8e26-07da5db475dc None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:50.248575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8363d9-1bb8-469d-8e26-07da5db475dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:50.275165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7869613d45064c3e9b07fd3d41183637', 'request_id': 'req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:50.305182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7869613d45064c3e9b07fd3d41183637', 'request_id': 'req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:50.309114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8c8363d9-1bb8-469d-8e26-07da5db475dc None None] MFA Rules not processed for user `766511b86ad643c099998fc4e1638f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:50.310837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8c8363d9-1bb8-469d-8e26-07da5db475dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:50.358980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] The token cache is being invalidate because project 7869613d45064c3e9b07fd3d41183637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:50.366240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:50.369082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-657bf5da-129a-4d4d-a69c-bfe0a1e2fd16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:50.378363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 869/1754] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:49 2022] DELETE /identity/v3/projects/7869613d45064c3e9b07fd3d41183637 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:50.383330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350adf-900c-40a0-a82f-fd744982ea02 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:50.383849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350adf-900c-40a0-a82f-fd744982ea02 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:50.384292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350adf-900c-40a0-a82f-fd744982ea02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:50.426061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d9350adf-900c-40a0-a82f-fd744982ea02 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:50.427681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9350adf-900c-40a0-a82f-fd744982ea02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:50.479767 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 886/1755] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:50.601301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 870/1756] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:50.609537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a4b44-97e3-4e78-acda-87b2a05f8128 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:50.610790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a4b44-97e3-4e78-acda-87b2a05f8128 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:50.610901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a4b44-97e3-4e78-acda-87b2a05f8128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:50.658710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a80a4b44-97e3-4e78-acda-87b2a05f8128 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:50.668979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a80a4b44-97e3-4e78-acda-87b2a05f8128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:50.694498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9350adf-900c-40a0-a82f-fd744982ea02 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:50.701602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 887/1757] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:50.706451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9350adf-900c-40a0-a82f-fd744982ea02 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:50.714732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:50.715594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:50.716093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:50.724392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:50.729222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:50.729670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:50.731382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:50.761365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a4b44-97e3-4e78-acda-87b2a05f8128 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:50.771244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 871/1758] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:50.776291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a4b44-97e3-4e78-acda-87b2a05f8128 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:50.787783 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:50.788686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:50.788968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:50.789244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:50.790108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:50.792576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:50.867702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:50.884426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 888/1759] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:50 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:50.894861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c480b7a-2078-45c4-ac72-e2dcb6320837 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:50.959411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:51.086511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:51.087331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:51.095731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:51.096389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:51.097682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:51.100404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:51.191851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:51.210110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 872/1760] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:50 2022] POST /identity/v3/users => generated 303 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:51.214759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d99d9-b038-48ba-b573-73acf4d1e7e1 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:51.236358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:51.236988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:51.237262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:51.237528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:51.238251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:51.246550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:51.253296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:51.304171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 889/1761] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:51.308533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:51.326951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf4217-8057-4ee9-84c9-e5407fdef0fb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:51.359129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4198d86-b8fa-4849-9459-7f51ccbde831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:51.360602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198d86-b8fa-4849-9459-7f51ccbde831 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:51.367181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198d86-b8fa-4849-9459-7f51ccbde831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:51.368205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198d86-b8fa-4849-9459-7f51ccbde831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:51.369222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4198d86-b8fa-4849-9459-7f51ccbde831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:51.374013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4198d86-b8fa-4849-9459-7f51ccbde831 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:51.398449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 873/1762] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:51.403589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:51.435922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e0ade-532b-4a16-a696-3fcba147073f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:51.455607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a6061-c9d6-4375-8a17-dbbc1e602368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:51.456623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a6061-c9d6-4375-8a17-dbbc1e602368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:51.456911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a6061-c9d6-4375-8a17-dbbc1e602368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:51.458864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a6061-c9d6-4375-8a17-dbbc1e602368 admin admin] PATH_INFO: `/v3/projects/187fcd6c746e4dd98fe6f83aa595503d/users/453cb70ac54142f69139437517082a9e/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:51.511448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a6061-c9d6-4375-8a17-dbbc1e602368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187fcd6c746e4dd98fe6f83aa595503d, user_id=453cb70ac54142f69139437517082a9e, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:51.516787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a6061-c9d6-4375-8a17-dbbc1e602368 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:51.571917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 890/1763] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:51 2022] PUT /identity/v3/projects/187fcd6c746e4dd98fe6f83aa595503d/users/453cb70ac54142f69139437517082a9e/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:51.591561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a003bd3-8104-4dbb-a09e-f3e6df4ea17a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:51.592229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a003bd3-8104-4dbb-a09e-f3e6df4ea17a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:51.592978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a003bd3-8104-4dbb-a09e-f3e6df4ea17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:51.655420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3a003bd3-8104-4dbb-a09e-f3e6df4ea17a None None] MFA Rules not processed for user `453cb70ac54142f69139437517082a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:51.657450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3a003bd3-8104-4dbb-a09e-f3e6df4ea17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:51.667254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42de8c-07c2-40e9-9776-468e42005fe7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:51.668419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42de8c-07c2-40e9-9776-468e42005fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:51.668914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42de8c-07c2-40e9-9776-468e42005fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:51.811737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ae42de8c-07c2-40e9-9776-468e42005fe7 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:51.813464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ae42de8c-07c2-40e9-9776-468e42005fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:51.850156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 874/1764] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:51 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:51.851128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a003bd3-8104-4dbb-a09e-f3e6df4ea17a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:52.017192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 891/1765] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:57:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:52.107906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a003bd3-8104-4dbb-a09e-f3e6df4ea17a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:52.123261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:52.123942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:52.124198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:52.124640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:52.150239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:52.152207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:52.292335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42de8c-07c2-40e9-9776-468e42005fe7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:52.294229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 875/1766] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:51 2022] POST /identity/v3/projects => generated 403 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:52.311757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42de8c-07c2-40e9-9776-468e42005fe7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:52.483673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:52.486856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:52.564498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:52.617979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:52.618784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:52.619124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:52.619406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:52.620064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:52.622260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:52.753111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 892/1767] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:52 2022] POST /identity/v3/users => generated 308 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:52.755966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:52.774861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7788-e432-4982-b89a-1f28822d92a8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:52.792358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:52.795331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:52.795996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:52.797018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:52.800114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:52.803984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:52.826096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 893/1768] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:52.831292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:52.874175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf6c8e-a233-4a58-b0ee-69ae531d7a17 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:52.875063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43053bd1-c6ad-4c7e-b2d8-1b224be7a5b6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:52.893824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:52.895909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:52.896463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:52.898517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:52.902084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:52.902774 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:52.903056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:52.903332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] PATH_INFO: `/v3/projects/5b41f5bbc73e46c68879f7a99527d8c1/users/7f30125dd89c4be4aa36d29403542995/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:52.963225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b41f5bbc73e46c68879f7a99527d8c1, user_id=7f30125dd89c4be4aa36d29403542995, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:52.966182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:52.979130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:52.991934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 894/1769] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:52 2022] PUT /identity/v3/projects/5b41f5bbc73e46c68879f7a99527d8c1/users/7f30125dd89c4be4aa36d29403542995/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:52.998516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:53.093481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:53.100378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:53.115792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 876/1770] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 11:57:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:57:53.176569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598845f-f2dd-4f47-9eda-e6776f5f3bbd admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:53.188875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492946-63d7-47e8-ac85-30516437872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:53.190043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492946-63d7-47e8-ac85-30516437872e admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:53.190571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492946-63d7-47e8-ac85-30516437872e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:53.191082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492946-63d7-47e8-ac85-30516437872e admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:53.192065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40492946-63d7-47e8-ac85-30516437872e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:53.193901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40492946-63d7-47e8-ac85-30516437872e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:53.227674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 895/1771] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:53.228350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:53.264144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b6209-dc59-4c3d-96ac-2be1d903fb86 service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:53.294724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:53.294724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:53.295009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:53.295303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] PATH_INFO: `/v3/projects/5b41f5bbc73e46c68879f7a99527d8c1/users/7f30125dd89c4be4aa36d29403542995/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:53.319108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b41f5bbc73e46c68879f7a99527d8c1, user_id=7f30125dd89c4be4aa36d29403542995, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:53.322079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:53.364737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 877/1772] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:53 2022] PUT /identity/v3/projects/5b41f5bbc73e46c68879f7a99527d8c1/users/7f30125dd89c4be4aa36d29403542995/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:53.377811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12f00-9554-4f6a-8fdb-3897aa8cde7a None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:53.378925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12f00-9554-4f6a-8fdb-3897aa8cde7a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:53.380601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12f00-9554-4f6a-8fdb-3897aa8cde7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:53.441393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2ee12f00-9554-4f6a-8fdb-3897aa8cde7a None None] MFA Rules not processed for user `7f30125dd89c4be4aa36d29403542995`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:53.446411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2ee12f00-9554-4f6a-8fdb-3897aa8cde7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:53.653968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 896/1773] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:53 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:53.668357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:53.671767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:53.881219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49cccc-40e3-4d5f-bfa6-d89be7caf2b5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:53.896918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a0328-14f7-4410-bba4-5013de422cdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:53.897719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a0328-14f7-4410-bba4-5013de422cdf service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:53.898136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a0328-14f7-4410-bba4-5013de422cdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:53.898530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a0328-14f7-4410-bba4-5013de422cdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:53.926703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a0328-14f7-4410-bba4-5013de422cdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:53.928163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a0328-14f7-4410-bba4-5013de422cdf service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:53.955289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 878/1774] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:53 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:55.480182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12f00-9554-4f6a-8fdb-3897aa8cde7a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:55.614065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12f00-9554-4f6a-8fdb-3897aa8cde7a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:55.618818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:55.619476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:55.619755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:55.620029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] PATH_INFO: `/v3/users/453cb70ac54142f69139437517082a9e` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:55.621196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] RBAC: Authorizing `identity:delete_user(user_id=453cb70ac54142f69139437517082a9e)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:55.623482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:55.657397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:55.717628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '453cb70ac54142f69139437517082a9e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:55.755281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '453cb70ac54142f69139437517082a9e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:55.755281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '453cb70ac54142f69139437517082a9e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:55.777477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '453cb70ac54142f69139437517082a9e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:55.815077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '453cb70ac54142f69139437517082a9e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:55.829199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 897/1775] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:55 2022] DELETE /identity/v3/users/453cb70ac54142f69139437517082a9e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:55.830694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:55.831204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:55.831313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:55.853663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:55.856864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:55.882657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:55.938626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 879/1776] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:55.951219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:55.953897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:56.054237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98ff5f-5726-4971-8d73-6668130af91b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:56.058317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:56.059157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:56.059323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:56.059620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:56.070675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:56.072224 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:56.083858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 898/1777] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:56.500094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:56.501651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:56.521900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c34f-e7ad-4721-a300-bc8c1d7f734a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:56.543359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:56.543359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:56.543613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:56.543868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] PATH_INFO: `/v3/projects/187fcd6c746e4dd98fe6f83aa595503d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:56.545424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=187fcd6c746e4dd98fe6f83aa595503d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:56.549800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:56.658901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187fcd6c746e4dd98fe6f83aa595503d', 'request_id': 'req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.668242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187fcd6c746e4dd98fe6f83aa595503d', 'request_id': 'req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.692619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187fcd6c746e4dd98fe6f83aa595503d', 'request_id': 'req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.712940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] The token cache is being invalidate because project 187fcd6c746e4dd98fe6f83aa595503d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:56.713272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.714365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.716220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 880/1778] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:56 2022] DELETE /identity/v3/projects/187fcd6c746e4dd98fe6f83aa595503d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:57:56.720518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:56.731597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87ebf-f72e-4800-bec9-c02646f64986 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:56.799395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:56.800061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:56.800345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:56.800661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] PATH_INFO: `/v3/users/7f30125dd89c4be4aa36d29403542995` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:56.802049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f30125dd89c4be4aa36d29403542995)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:56.804008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:56.907488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f30125dd89c4be4aa36d29403542995'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.910303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:56.926795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f30125dd89c4be4aa36d29403542995'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.930089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f30125dd89c4be4aa36d29403542995'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.942211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f30125dd89c4be4aa36d29403542995'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.978459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b71b2715-54c9-4be6-9f97-de4985cab7a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f30125dd89c4be4aa36d29403542995'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:56.986449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 899/1779] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:56 2022] DELETE /identity/v3/users/7f30125dd89c4be4aa36d29403542995 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:57.504632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:57.507040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:57.530988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0093d-8167-45d1-9f1a-0e4b95ee82d4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:57.649723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:57.650545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:57.650896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:57.651194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] PATH_INFO: `/v3/projects/5b41f5bbc73e46c68879f7a99527d8c1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:57.652614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b41f5bbc73e46c68879f7a99527d8c1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:57.654378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:57.768637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b41f5bbc73e46c68879f7a99527d8c1', 'request_id': 'req-b55f15e2-58fe-4640-8f8d-1a047eba3167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:57.772805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:57.781231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b41f5bbc73e46c68879f7a99527d8c1', 'request_id': 'req-b55f15e2-58fe-4640-8f8d-1a047eba3167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:57.806475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b41f5bbc73e46c68879f7a99527d8c1', 'request_id': 'req-b55f15e2-58fe-4640-8f8d-1a047eba3167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:57.837939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] The token cache is being invalidate because project 5b41f5bbc73e46c68879f7a99527d8c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:57.838420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:57.839777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b55f15e2-58fe-4640-8f8d-1a047eba3167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:57.842467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 881/1780] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:57 2022] DELETE /identity/v3/projects/5b41f5bbc73e46c68879f7a99527d8c1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:57.856938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4b6f9-1c84-4fac-88b6-279e82ae3687 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:57.857533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4b6f9-1c84-4fac-88b6-279e82ae3687 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:57.857918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4b6f9-1c84-4fac-88b6-279e82ae3687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:57.900772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0bc4b6f9-1c84-4fac-88b6-279e82ae3687 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:57.902269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0bc4b6f9-1c84-4fac-88b6-279e82ae3687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:58.045831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 900/1781] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:58.051744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aefb9-bd02-483b-ad96-344865a9efe2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.052236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aefb9-bd02-483b-ad96-344865a9efe2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.052721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aefb9-bd02-483b-ad96-344865a9efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.105321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-181aefb9-bd02-483b-ad96-344865a9efe2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:58.106610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-181aefb9-bd02-483b-ad96-344865a9efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:58.118434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 882/1782] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:58.123640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4b6f9-1c84-4fac-88b6-279e82ae3687 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:58.152382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4b6f9-1c84-4fac-88b6-279e82ae3687 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:58.162866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:58.163504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.163842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.164115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.164792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:58.165094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:58.181724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:58.201272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 901/1783] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:58.205946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181aefb9-bd02-483b-ad96-344865a9efe2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:58.217450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181aefb9-bd02-483b-ad96-344865a9efe2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:58.230725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:58.231799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.232157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.232439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.233278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:58.236183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:58.293838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 883/1784] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:58 2022] POST /identity/v3/projects => generated 399 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:57:58.300970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:58.320303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3309-96d2-442e-9944-68ea541340e5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:58.402585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:58.403214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.403537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.403796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.404433 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:58.406495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:58.436115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 902/1785] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:58 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:57:58.441228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:58.452705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d6b9a-d200-4f7b-a40d-1ec4481824b0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:58.457721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:58.458402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.458649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.458955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.459608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:58.461303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:58.495796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 884/1786] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:57:58.496985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:58.513412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189ee0e-05b1-445a-8dec-b8ed93feab38 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:58.535334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ad286-15b0-45a9-8b7d-9c6bd7a68258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:58.536950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad286-15b0-45a9-8b7d-9c6bd7a68258 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.537439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad286-15b0-45a9-8b7d-9c6bd7a68258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.537946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad286-15b0-45a9-8b7d-9c6bd7a68258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.539579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ad286-15b0-45a9-8b7d-9c6bd7a68258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:58.542800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ad286-15b0-45a9-8b7d-9c6bd7a68258 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:58.565921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 903/1787] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:57:58.567709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:58.589701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52359cf2-2e1a-4043-ba56-9e6087fa18ac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:58.600178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:58.602278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.603904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.613364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] PATH_INFO: `/v3/projects/da0a2de7b6124b57837351d10ece2b83/users/15c7df31bb6c44b9992e89c228f4b7ec/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.629153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=da0a2de7b6124b57837351d10ece2b83, user_id=15c7df31bb6c44b9992e89c228f4b7ec, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:58.632817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:58.663332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 885/1788] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:58 2022] PUT /identity/v3/projects/da0a2de7b6124b57837351d10ece2b83/users/15c7df31bb6c44b9992e89c228f4b7ec/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:58.666705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e31ee4-e864-4bbc-b0ff-a6c8b8e53ce0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:58.667038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e31ee4-e864-4bbc-b0ff-a6c8b8e53ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:58.667674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e31ee4-e864-4bbc-b0ff-a6c8b8e53ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:58.705182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c3e31ee4-e864-4bbc-b0ff-a6c8b8e53ce0 None None] MFA Rules not processed for user `15c7df31bb6c44b9992e89c228f4b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:58.706524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c3e31ee4-e864-4bbc-b0ff-a6c8b8e53ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:58.723929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:58.853483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 904/1789] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:58.858992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:58.998141 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601ef16-3e01-4dc3-88fd-4936455cd8ef admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:59.004304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:59.005268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:59.005403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:59.005608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:59.006316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:59.008758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:59.049980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 886/1790] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:58 2022] POST /identity/v3/projects => generated 397 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:59.055665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e31ee4-e864-4bbc-b0ff-a6c8b8e53ce0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:59.075435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e31ee4-e864-4bbc-b0ff-a6c8b8e53ce0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:59.260628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:59.262072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:59.262725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:59.263343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:59.264554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:59.308248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:59.377879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:59.381062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 905/1791] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:59 2022] POST /identity/v3/users => generated 305 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:57:59.396803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893b354-5521-41fb-87d7-e326175f7dca admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:59.422284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d34576-9f2c-4bde-8267-f406b899fa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:59.423029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34576-9f2c-4bde-8267-f406b899fa58 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:59.428129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34576-9f2c-4bde-8267-f406b899fa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:59.428129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34576-9f2c-4bde-8267-f406b899fa58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:59.445656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d34576-9f2c-4bde-8267-f406b899fa58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:59.451350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d34576-9f2c-4bde-8267-f406b899fa58 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:59.522022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 887/1792] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:59.526702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:59.543066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd20796-a679-4bb1-841c-9a3a2627fd4a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:59.543373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:59.543808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:59.544216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:59.561170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:59.561941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:59.562231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:59.565162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] PATH_INFO: `/v3/projects/5f8bb1dfaf2849d493c1f157057fcb64/users/326c0b1b0a184ef7ae2a4c27ade84cfc/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:59.607928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] MFA Rules not processed for user `dee035c97cb6447382365faf6b6aca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:59.609953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:59.652183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8bb1dfaf2849d493c1f157057fcb64, user_id=326c0b1b0a184ef7ae2a4c27ade84cfc, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:59.660247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:59.761686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 906/1793] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:59 2022] PUT /identity/v3/projects/5f8bb1dfaf2849d493c1f157057fcb64/users/326c0b1b0a184ef7ae2a4c27ade84cfc/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:59.775650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:59.796843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:59.823489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 888/1794] 10.222.0.87 () {58 vars in 1015 bytes} [Mon Sep 19 11:57:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:59.933882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:59.936845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:59.968314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6238a-05c4-4b33-abe9-11031268fde9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:00.097846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eefed-2259-435d-9df5-fef45f3b3310 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:00.115190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:00.117266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:00.117976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:00.118424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:00.120845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:00.124298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:00.162496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d64a3-844a-4c2c-aede-77d10e378514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:00.163152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d64a3-844a-4c2c-aede-77d10e378514 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:00.163440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d64a3-844a-4c2c-aede-77d10e378514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:00.163713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d64a3-844a-4c2c-aede-77d10e378514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:00.180730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 907/1795] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:00.184609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:00.198483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d64a3-844a-4c2c-aede-77d10e378514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:00.211919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d64a3-844a-4c2c-aede-77d10e378514 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:00.233512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8df48-5bc6-4291-86af-d361fe3139ba admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:00.255461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:00.256725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:00.258037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:00.259059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] PATH_INFO: `/v3/projects/5f8bb1dfaf2849d493c1f157057fcb64/users/326c0b1b0a184ef7ae2a4c27ade84cfc/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:00.270625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 889/1796] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:00.291029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8bb1dfaf2849d493c1f157057fcb64, user_id=326c0b1b0a184ef7ae2a4c27ade84cfc, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:00.318500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:00.402591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 908/1797] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:00 2022] PUT /identity/v3/projects/5f8bb1dfaf2849d493c1f157057fcb64/users/326c0b1b0a184ef7ae2a4c27ade84cfc/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:00.425619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:00.426494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:00.426991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:00.544606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] MFA Rules not processed for user `326c0b1b0a184ef7ae2a4c27ade84cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:00.550957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:00.590206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:00.592492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:00.611171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:00.834248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68e3f-96d8-4835-ad7d-e53dfa07ab67 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:00.855107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:00.856305 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:00.858970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:00.861320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] PATH_INFO: `/v3/users/f90ac5451757467999b63c178e135db1` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:00.872639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90ac5451757467999b63c178e135db1)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:00.884960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:00.899561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 890/1798] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:00 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:00.909170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:00.918002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:00.973794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74223-8cd9-4856-9744-e26ad266a474 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:00.987185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:00.987978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:00.988521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:00.989009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] PATH_INFO: `/v3/users/cfe2be353f0c409c82e1e269e25cf34c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:00.991764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe2be353f0c409c82e1e269e25cf34c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:00.994241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:01.053149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90ac5451757467999b63c178e135db1'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.070538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90ac5451757467999b63c178e135db1'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.071477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90ac5451757467999b63c178e135db1'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.108645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90ac5451757467999b63c178e135db1'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.163250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe2be353f0c409c82e1e269e25cf34c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.163793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe2be353f0c409c82e1e269e25cf34c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.174927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90ac5451757467999b63c178e135db1'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.201298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe2be353f0c409c82e1e269e25cf34c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.209057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 909/1799] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:00 2022] DELETE /identity/v3/users/f90ac5451757467999b63c178e135db1 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:01.211702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:01.223008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:01.235244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe2be353f0c409c82e1e269e25cf34c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.353724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2781ed88-6a96-45ae-b4bf-e8654b32f9f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe2be353f0c409c82e1e269e25cf34c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:01.371044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:01.391636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 891/1800] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:00 2022] DELETE /identity/v3/users/cfe2be353f0c409c82e1e269e25cf34c => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:01.393239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c928-d657-45ac-aed6-fad72ac12fcf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:01.393525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c928-d657-45ac-aed6-fad72ac12fcf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:01.393816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c928-d657-45ac-aed6-fad72ac12fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:01.513304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2682c928-d657-45ac-aed6-fad72ac12fcf None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:01.530856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2682c928-d657-45ac-aed6-fad72ac12fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:01.558744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084f463-116c-4cc8-8cc2-028b6081a1ea admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:01.676093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:01.715953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:01.736010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:01.736366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:01.736673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:01.744230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:01.813879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 892/1801] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:01.814784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:01.814784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:01.815023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:01.869382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:01.870935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:01.894189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 893/1802] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:01.917251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:01.919036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:01.944345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 910/1803] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:01 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:02.021635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:02.024179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:02.077712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:02.080667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:02.211250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae95f0-fc67-4180-82d7-4a649d577b56 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:02.240132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:02.240806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:02.241087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:02.241524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:02.269892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:02.272690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:02.292251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53d6dc-7e91-4c56-8716-1189ac032fe2 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:02.293886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 894/1804] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:02.294767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:02.296298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:02.301160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:02.303985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:02.304708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:02.306265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:02.312206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f4507-2fed-42a9-8cf2-e431dcf55ce6 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:02.317881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:02.324374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:02.324374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:02.324374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:02.324374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:02.365069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:02.372165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:02.421000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 911/1805] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:02.438778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:02.440443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:02.482385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 895/1806] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:04.898124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:04.903170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:04.968213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048ec87-bcbe-4c71-a149-2478d1ee7144 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:04.983320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:04.983992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:04.984272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:04.987140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] PATH_INFO: `/v3/projects/075fcaf37ea74f2f987976d9ee2b8d8c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:04.990083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=075fcaf37ea74f2f987976d9ee2b8d8c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:04.991942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:05.113308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '075fcaf37ea74f2f987976d9ee2b8d8c', 'request_id': 'req-f7f4e23b-569d-40d1-b9f7-25987d255c12'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.139751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '075fcaf37ea74f2f987976d9ee2b8d8c', 'request_id': 'req-f7f4e23b-569d-40d1-b9f7-25987d255c12'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.152647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '075fcaf37ea74f2f987976d9ee2b8d8c', 'request_id': 'req-f7f4e23b-569d-40d1-b9f7-25987d255c12'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.182125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] The token cache is being invalidate because project 075fcaf37ea74f2f987976d9ee2b8d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:05.182751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.183786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.190335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 912/1807] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:04 2022] DELETE /identity/v3/projects/075fcaf37ea74f2f987976d9ee2b8d8c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:05.194693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:05.215049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1043d-94d1-4e0c-a290-15e7da4fbfd3 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:05.327519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:05.402578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:05.404576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:05.406309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:05.407827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] PATH_INFO: `/v3/users/b16445c9b7e9499c95029d3ff27efc1a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:05.414314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16445c9b7e9499c95029d3ff27efc1a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:05.425035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:05.604400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16445c9b7e9499c95029d3ff27efc1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.608361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16445c9b7e9499c95029d3ff27efc1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.701944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16445c9b7e9499c95029d3ff27efc1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.736211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16445c9b7e9499c95029d3ff27efc1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.789886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0e103a23-adb9-4275-abfc-cd395add3259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16445c9b7e9499c95029d3ff27efc1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:05.814370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 896/1808] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:05 2022] DELETE /identity/v3/users/b16445c9b7e9499c95029d3ff27efc1a => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:05.950265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:05.951995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:06.001203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4e23b-569d-40d1-b9f7-25987d255c12 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:06.034692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:06.138839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:06.139791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:06.140342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:06.140894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] PATH_INFO: `/v3/projects/0b356e0c6337455593a3441e44d93790` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:06.143852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b356e0c6337455593a3441e44d93790)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:06.146274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:06.174367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafb1da-8a65-481d-aaa1-44d6fc80cdda None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:06.175790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafb1da-8a65-481d-aaa1-44d6fc80cdda None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:06.175929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafb1da-8a65-481d-aaa1-44d6fc80cdda None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:06.178501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 897/1809] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:58:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:58:06.211001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:06.213481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:06.215138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:06.232295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b356e0c6337455593a3441e44d93790', 'request_id': 'req-c867d1de-69fd-46d9-acae-dc619758b5de'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.245907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b356e0c6337455593a3441e44d93790', 'request_id': 'req-c867d1de-69fd-46d9-acae-dc619758b5de'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.258502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b356e0c6337455593a3441e44d93790', 'request_id': 'req-c867d1de-69fd-46d9-acae-dc619758b5de'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.269356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:06.272318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:06.275786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] The token cache is being invalidate because project 0b356e0c6337455593a3441e44d93790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:06.276672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.277641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.280692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 913/1810] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:05 2022] DELETE /identity/v3/projects/0b356e0c6337455593a3441e44d93790 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:06.289325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:06.291088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:06.442841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867d1de-69fd-46d9-acae-dc619758b5de admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:06.451459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:06.451866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 898/1811] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:06.452392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:06.453812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:06.453982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] PATH_INFO: `/v3/users/bc0993c3236744208a86ab5e1796ec2d` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:06.455440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc0993c3236744208a86ab5e1796ec2d)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:06.457891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:06.524988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:06.529733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc0993c3236744208a86ab5e1796ec2d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.530080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:06.532917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:06.547752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc0993c3236744208a86ab5e1796ec2d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.549241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc0993c3236744208a86ab5e1796ec2d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.554303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a7d0a-2a0d-4d82-96f8-69c7f17bea07 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:06.561095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b3cb1-d682-4f47-9c7e-336ea5cbfc0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:06.562124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b3cb1-d682-4f47-9c7e-336ea5cbfc0f service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:06.563543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b3cb1-d682-4f47-9c7e-336ea5cbfc0f service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:06.564582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b3cb1-d682-4f47-9c7e-336ea5cbfc0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:06.566976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e4b3cb1-d682-4f47-9c7e-336ea5cbfc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:06.605365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc0993c3236744208a86ab5e1796ec2d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.605970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b3cb1-d682-4f47-9c7e-336ea5cbfc0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:06.607958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b3cb1-d682-4f47-9c7e-336ea5cbfc0f service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:06.630454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 899/1812] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:58:06.639262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-813e074d-0d4a-4c1f-955a-81434e2c9421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc0993c3236744208a86ab5e1796ec2d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:06.655023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 914/1813] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:06 2022] DELETE /identity/v3/users/bc0993c3236744208a86ab5e1796ec2d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:06.969192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769e39d-148c-4c58-a1e2-9a8ee20bfc2a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:06.969890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769e39d-148c-4c58-a1e2-9a8ee20bfc2a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:06.970825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769e39d-148c-4c58-a1e2-9a8ee20bfc2a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:06.972970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 900/1814] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:58:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:58:07.039691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:07.040554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:07.040999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:07.105702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:07.107057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:07.263200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 915/1815] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:58:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:07.414456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769e39d-148c-4c58-a1e2-9a8ee20bfc2a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:07.417142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9769e39d-148c-4c58-a1e2-9a8ee20bfc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:07.487427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769e39d-148c-4c58-a1e2-9a8ee20bfc2a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:07.510059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd25cc6-9179-4538-8249-231865ef7cf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:07.514055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25cc6-9179-4538-8249-231865ef7cf0 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:07.518254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25cc6-9179-4538-8249-231865ef7cf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:07.518557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25cc6-9179-4538-8249-231865ef7cf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:07.532670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:07.534787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:07.559116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd25cc6-9179-4538-8249-231865ef7cf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:07.560734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd25cc6-9179-4538-8249-231865ef7cf0 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:07.581496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 901/1816] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:07.704531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f51de-b518-4a3f-98ad-b26970afc6b7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:07.711890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:07.712773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:07.713115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:07.713499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] PATH_INFO: `/v3/projects/4b7de59eda3848cf9ce1053f585327cb` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:07.715416 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7de59eda3848cf9ce1053f585327cb)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:07.717754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:07.783145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7de59eda3848cf9ce1053f585327cb', 'request_id': 'req-019191a5-bd47-428d-82ac-b498ea6f2118'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:07.800833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7de59eda3848cf9ce1053f585327cb', 'request_id': 'req-019191a5-bd47-428d-82ac-b498ea6f2118'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:07.805475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:07.813673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7de59eda3848cf9ce1053f585327cb', 'request_id': 'req-019191a5-bd47-428d-82ac-b498ea6f2118'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:07.852851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] The token cache is being invalidate because project 4b7de59eda3848cf9ce1053f585327cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:07.853716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:07.854732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-019191a5-bd47-428d-82ac-b498ea6f2118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:07.862018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7398d-d037-4986-ad3d-3bc704bb385a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:07.862520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 916/1817] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:07 2022] DELETE /identity/v3/projects/4b7de59eda3848cf9ce1053f585327cb => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:07.862673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7398d-d037-4986-ad3d-3bc704bb385a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:07.863117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7398d-d037-4986-ad3d-3bc704bb385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:07.892595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b7f7398d-d037-4986-ad3d-3bc704bb385a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:07.894122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b7f7398d-d037-4986-ad3d-3bc704bb385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:07.910186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:08.004908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 902/1818] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:08.012622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cc83e-3e4f-4517-bfa9-8568f842fac7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.012900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cc83e-3e4f-4517-bfa9-8568f842fac7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.013173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cc83e-3e4f-4517-bfa9-8568f842fac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.034345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-454cc83e-3e4f-4517-bfa9-8568f842fac7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:08.035927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-454cc83e-3e4f-4517-bfa9-8568f842fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:08.048201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 917/1819] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:08.055042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7398d-d037-4986-ad3d-3bc704bb385a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.066353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7398d-d037-4986-ad3d-3bc704bb385a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.072492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.073548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.073968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.074521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.075769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:08.076863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.079284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.093497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 903/1820] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:08.097803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cc83e-3e4f-4517-bfa9-8568f842fac7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.107635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cc83e-3e4f-4517-bfa9-8568f842fac7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.112716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.113331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.113611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.113926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.114575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.116487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.153893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 918/1821] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:08 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:08.158673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.168522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dc90f-f470-4085-b662-2bcc151f19d1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.229586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.230250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.230564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.230841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.231480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.233427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.258506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 904/1822] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:08 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:08.262736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.274112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed8e31-bf4d-41bb-9630-322fa8a91489 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.279341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.280054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.280400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.280738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.282070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.283749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.300326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 919/1823] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:08.305295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.322623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898ae48-953b-42ea-8982-9f2830a2b6c4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.334597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.335309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.335660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.336263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.337281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.339889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.364308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 905/1824] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:08.366616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.380582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.383981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:08.391047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cefc5-4b78-46f0-81ff-ee39bfac7544 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.399157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8cca0-5d80-452d-b76f-2c1208635ad9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.404479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.405132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.405414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.405691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] PATH_INFO: `/v3/projects/405638a816464e0ca2a0808c235950b7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.408711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314c8bc-800c-4ef4-b4e8-ec74e02f5ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.409925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314c8bc-800c-4ef4-b4e8-ec74e02f5ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.411797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314c8bc-800c-4ef4-b4e8-ec74e02f5ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.413924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314c8bc-800c-4ef4-b4e8-ec74e02f5ea9 admin admin] PATH_INFO: `/v3/projects/7fc73cc5c64b428cb1ddc98fbc406ff8/users/b1823b16383d4a26a4d638cf3670457e/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.418668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=405638a816464e0ca2a0808c235950b7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.427334 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.507950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314c8bc-800c-4ef4-b4e8-ec74e02f5ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc73cc5c64b428cb1ddc98fbc406ff8, user_id=b1823b16383d4a26a4d638cf3670457e, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.515195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314c8bc-800c-4ef4-b4e8-ec74e02f5ea9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.556958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 920/1825] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:08 2022] PUT /identity/v3/projects/7fc73cc5c64b428cb1ddc98fbc406ff8/users/b1823b16383d4a26a4d638cf3670457e/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:08.558958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.559825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.562346 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.586808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405638a816464e0ca2a0808c235950b7', 'request_id': 'req-817776fa-f705-4f65-bd37-623926d1ab23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:08.620809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405638a816464e0ca2a0808c235950b7', 'request_id': 'req-817776fa-f705-4f65-bd37-623926d1ab23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:08.650579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405638a816464e0ca2a0808c235950b7', 'request_id': 'req-817776fa-f705-4f65-bd37-623926d1ab23'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:08.686572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] The token cache is being invalidate because project 405638a816464e0ca2a0808c235950b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:08.686734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:08.687830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-817776fa-f705-4f65-bd37-623926d1ab23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:08.690788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] MFA Rules not processed for user `b1823b16383d4a26a4d638cf3670457e`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:08.694270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:08.702456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 906/1826] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:08 2022] DELETE /identity/v3/projects/405638a816464e0ca2a0808c235950b7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:08.709878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.710483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.710952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.766906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:08.768347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:08.777350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:08.826470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 921/1827] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:08.827493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.829959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:08.892067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 907/1828] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:08.892067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:08.901127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:08.925331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f047744-f789-44a0-8a6e-4c133e201b35 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.932120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21daa4a4-682e-4e87-9fe5-6eb9007fa727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.933088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa4a4-682e-4e87-9fe5-6eb9007fa727 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.933534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa4a4-682e-4e87-9fe5-6eb9007fa727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.934183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa4a4-682e-4e87-9fe5-6eb9007fa727 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.935402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21daa4a4-682e-4e87-9fe5-6eb9007fa727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.937967 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21daa4a4-682e-4e87-9fe5-6eb9007fa727 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.959749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73585cf2-c87e-497d-bc80-411e7a02a866 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:08.965862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:08.966519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.966821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.967099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] PATH_INFO: `/v3/users/15c7df31bb6c44b9992e89c228f4b7ec` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.968317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=15c7df31bb6c44b9992e89c228f4b7ec)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:08.970365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:08.973809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 908/1829] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:08 2022] POST /identity/v3/projects => generated 389 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:08.986958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:08.987275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:08.987662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:08.991966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:09.012743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:09.014214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:09.040858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15c7df31bb6c44b9992e89c228f4b7ec'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:09.051553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15c7df31bb6c44b9992e89c228f4b7ec'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:09.052811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15c7df31bb6c44b9992e89c228f4b7ec'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:09.083863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15c7df31bb6c44b9992e89c228f4b7ec'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:09.096946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 909/1830] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:09.098036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.099606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:09.100687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15c7df31bb6c44b9992e89c228f4b7ec'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:09.113294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 922/1831] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:08 2022] DELETE /identity/v3/users/15c7df31bb6c44b9992e89c228f4b7ec => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:09.114634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.115523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13ae68-c47f-435a-a54d-dd226cd5001b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.185619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa6a5f-7b1a-41af-bbb3-a99d73fddf1f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.191731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8259d94-0a82-4550-9dc5-186f2db5c84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.192494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8259d94-0a82-4550-9dc5-186f2db5c84b admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.193021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8259d94-0a82-4550-9dc5-186f2db5c84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.193363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8259d94-0a82-4550-9dc5-186f2db5c84b admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.194213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8259d94-0a82-4550-9dc5-186f2db5c84b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:09.194588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8259d94-0a82-4550-9dc5-186f2db5c84b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.194911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.195549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.195824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.196089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.196574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8259d94-0a82-4550-9dc5-186f2db5c84b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.196883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.199162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.213991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 923/1832] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:09.216166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30aad-a40c-43fb-99bd-4c9b4742a9e7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.216506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30aad-a40c-43fb-99bd-4c9b4742a9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.216930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30aad-a40c-43fb-99bd-4c9b4742a9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.229323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 910/1833] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:09 2022] POST /identity/v3/users => generated 301 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:09.230754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.243186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e8b30aad-a40c-43fb-99bd-4c9b4742a9e7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:09.244343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e8b30aad-a40c-43fb-99bd-4c9b4742a9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:09.265799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092ce84-1af5-41c4-88ac-ffcb46e125d8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.266543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 924/1834] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:09.266543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30aad-a40c-43fb-99bd-4c9b4742a9e7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.268495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.269381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.269755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.270192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.270747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30aad-a40c-43fb-99bd-4c9b4742a9e7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.271292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.273283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.278119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.278776 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.279188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.279468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.280121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.282438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.292541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:09.295361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 925/1835] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:09.295361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.299384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:09.313833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 911/1836] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:09 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:09.315486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.380550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37351e5-e730-4725-af32-0171ffb1ff74 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.393098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458af8-0421-4c05-9785-5ce2fce566da admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.398275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.399025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.399374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.399635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] PATH_INFO: `/v3/projects/7b3696abdf1943de8c88cb5c45834fdd/users/ef92911721ae454bbf372bc519a87e8b/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.404037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:09.418840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3696abdf1943de8c88cb5c45834fdd, user_id=ef92911721ae454bbf372bc519a87e8b, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.423261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.438856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.439632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.439907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 912/1837] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:09 2022] PUT /identity/v3/projects/7b3696abdf1943de8c88cb5c45834fdd/users/ef92911721ae454bbf372bc519a87e8b/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:09.440178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.440717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.440954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.453207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.454669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.464761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 926/1838] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:09.465599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.476468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59047e-bdad-4e9a-9e81-56f58c06660e service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.529596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a2066-55ed-4157-b569-2618537b6ac4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.536720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.537698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.538320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.538441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.539648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.541752 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.545714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.546638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.547650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.548105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.549013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.551137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.563446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 927/1839] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:09.568913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 913/1840] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:09 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:09.570212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.573140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.580351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05a6c1-9d90-40c5-abbb-0a5334effb29 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.585435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c04f4e-b2bd-4c4f-9dad-dbcc7173baae admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.585797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacda8f6-f897-4e50-9008-406e1a703ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.586623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacda8f6-f897-4e50-9008-406e1a703ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.586928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacda8f6-f897-4e50-9008-406e1a703ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.587267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacda8f6-f897-4e50-9008-406e1a703ae3 admin admin] PATH_INFO: `/v3/projects/7b3696abdf1943de8c88cb5c45834fdd/users/ef92911721ae454bbf372bc519a87e8b/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.589535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacda8f6-f897-4e50-9008-406e1a703ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3696abdf1943de8c88cb5c45834fdd, user_id=ef92911721ae454bbf372bc519a87e8b, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.590735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.591387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.591719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.591996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.592654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.592846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacda8f6-f897-4e50-9008-406e1a703ae3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.594365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.605357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 914/1841] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:09.610598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.611991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 928/1842] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:09 2022] PUT /identity/v3/projects/7b3696abdf1943de8c88cb5c45834fdd/users/ef92911721ae454bbf372bc519a87e8b/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:09.624046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.626324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.626652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.632677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32bb13-b8af-467e-a104-8d67c53085ff admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.670517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] MFA Rules not processed for user `ef92911721ae454bbf372bc519a87e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:09.672313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:09.693304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:09.702497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.703725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.704394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.704864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.705904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.707952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.719402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 915/1843] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:09.724652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.738772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58a3f4-cc67-4c34-b91b-5673f39198e1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.745324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f3304-fa43-47c9-968b-fba0d5a8ee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.746068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3304-fa43-47c9-968b-fba0d5a8ee81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.746348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3304-fa43-47c9-968b-fba0d5a8ee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.746620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3304-fa43-47c9-968b-fba0d5a8ee81 admin admin] PATH_INFO: `/v3/projects/dddeef41a10e4939a923e806076aee3a/users/5e0fa1a5cb4645e4977fea4f34265c8c/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.766523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f3304-fa43-47c9-968b-fba0d5a8ee81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dddeef41a10e4939a923e806076aee3a, user_id=5e0fa1a5cb4645e4977fea4f34265c8c, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:09.769601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f3304-fa43-47c9-968b-fba0d5a8ee81 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:09.792513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 916/1844] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:09 2022] PUT /identity/v3/projects/dddeef41a10e4939a923e806076aee3a/users/5e0fa1a5cb4645e4977fea4f34265c8c/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:09.792741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 929/1845] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:09.798428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.798745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.799116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:09.806946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:09.808888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:09.855804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] MFA Rules not processed for user `5e0fa1a5cb4645e4977fea4f34265c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:09.857086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:09.868683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:09.893707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a00985-7d35-44a6-9753-24a0f7917d7f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:09.986062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:09.986940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:09.987728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:09.987977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.025108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 917/1846] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:10.031631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.033152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:10.114572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.118178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:10.138144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45f1a3-2d80-41f4-8004-11f5f2d66133 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:10.147025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 930/1847] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:58:10.147975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.151922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:10.152815 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.153233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.154220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] PATH_INFO: `/v3/projects/da0a2de7b6124b57837351d10ece2b83` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.156963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da0a2de7b6124b57837351d10ece2b83)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.159911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:10.163301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e9001-c135-4ddb-a1b6-fc0c1c2fbc44 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:10.178077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:10.178077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.178077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.178077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.178661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.179980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:10.235711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 931/1848] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:10 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:10.240619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.242444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da0a2de7b6124b57837351d10ece2b83', 'request_id': 'req-9be926d6-1de8-4fe2-a691-794f0ff314c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.254245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da0a2de7b6124b57837351d10ece2b83', 'request_id': 'req-9be926d6-1de8-4fe2-a691-794f0ff314c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.262725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:10.264293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da0a2de7b6124b57837351d10ece2b83', 'request_id': 'req-9be926d6-1de8-4fe2-a691-794f0ff314c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.286886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] The token cache is being invalidate because project da0a2de7b6124b57837351d10ece2b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:10.287316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.288561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.293843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 918/1849] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:10 2022] DELETE /identity/v3/projects/da0a2de7b6124b57837351d10ece2b83 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:10.294877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.296653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:10.392435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d83f-c942-4407-9b3a-c36395cee768 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:10.398417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:10.399066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.399722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.400199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.402287 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.404763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:10.438328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 932/1850] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:10 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:10.440059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.442298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:10.456664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be926d6-1de8-4fe2-a691-794f0ff314c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:10.457356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b303b-dd4f-4cc6-bf13-4dfeab985f22 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:10.466358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:10.469966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.469966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.469966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.469966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.470590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:10.470590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.470590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.470590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] PATH_INFO: `/v3/users/326c0b1b0a184ef7ae2a4c27ade84cfc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.472513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:10.475707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:10.475914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326c0b1b0a184ef7ae2a4c27ade84cfc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.478693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:10.492014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 933/1851] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:10.493019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.495149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:10.509798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520a008-3d77-49d9-bc43-4bc5abab4d3e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:10.521596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:10.522328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.522610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.522882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] PATH_INFO: `/v3/users/b1823b16383d4a26a4d638cf3670457e` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.524337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1823b16383d4a26a4d638cf3670457e)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.532273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:10.587798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326c0b1b0a184ef7ae2a4c27ade84cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.593418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326c0b1b0a184ef7ae2a4c27ade84cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.625824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326c0b1b0a184ef7ae2a4c27ade84cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.640492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1823b16383d4a26a4d638cf3670457e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.643087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326c0b1b0a184ef7ae2a4c27ade84cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.651491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1823b16383d4a26a4d638cf3670457e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.652670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1823b16383d4a26a4d638cf3670457e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.666134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1823b16383d4a26a4d638cf3670457e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.683810 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-b5c6b539-7137-42ec-9f7e-93ace4a4f0fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1823b16383d4a26a4d638cf3670457e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.686636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326c0b1b0a184ef7ae2a4c27ade84cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:10.697375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 919/1852] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:10 2022] DELETE /identity/v3/users/326c0b1b0a184ef7ae2a4c27ade84cfc => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:10.698437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.705950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:10.711832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 934/1853] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:10 2022] DELETE /identity/v3/users/b1823b16383d4a26a4d638cf3670457e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:10.716699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.716900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.717116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.743064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:10.744806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:10.802573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe4e5d-a910-44c5-95b9-c54089964a41 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:10.830021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:10.903100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 935/1854] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:10.918927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:10.920362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:10.933695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:10.934580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:10.935244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:10.935676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] PATH_INFO: `/v3/projects/35151454e49a4f4eb31507575b46325b/users/df0d947771d9419c98a2e42bb0a28df4/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:10.978952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35151454e49a4f4eb31507575b46325b, user_id=df0d947771d9419c98a2e42bb0a28df4, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:10.983601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:11.010030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 920/1855] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:10 2022] PUT /identity/v3/projects/35151454e49a4f4eb31507575b46325b/users/df0d947771d9419c98a2e42bb0a28df4/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:11.011877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:11.026014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:11.052939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5d70-db22-4f92-891e-6eee009efcb0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:11.106927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:11.172561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e39448-9e8a-402c-b7bf-46f6830f2ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:11.173283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e39448-9e8a-402c-b7bf-46f6830f2ada service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:11.174491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e39448-9e8a-402c-b7bf-46f6830f2ada service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:11.174794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e39448-9e8a-402c-b7bf-46f6830f2ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:11.220686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8a5cc-fdf5-449e-8688-b4ca19fca7e0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:11.232970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:11.234974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:11.236212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:11.236472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:11.237236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:11.243517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:11.300047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 921/1856] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:11.301569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:11.307017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e39448-9e8a-402c-b7bf-46f6830f2ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:11.320445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf13aaf-0147-4342-b956-3e216ccd93af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:11.320932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e39448-9e8a-402c-b7bf-46f6830f2ada service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:11.335507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:11.338006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:11.339470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:11.341169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] PATH_INFO: `/v3/projects/35151454e49a4f4eb31507575b46325b/users/df0d947771d9419c98a2e42bb0a28df4/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:11.347606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] RBAC: Authorizing `identity:create_grant(project_id=35151454e49a4f4eb31507575b46325b, user_id=df0d947771d9419c98a2e42bb0a28df4, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:11.369146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:11.386120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 936/1857] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:58:11.425923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 922/1858] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:11 2022] PUT /identity/v3/projects/35151454e49a4f4eb31507575b46325b/users/df0d947771d9419c98a2e42bb0a28df4/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:11.431902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:11.432210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:11.432490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:11.480603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] MFA Rules not processed for user `df0d947771d9419c98a2e42bb0a28df4`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:11.482514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:11.611210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:11.613113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:11.628719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 937/1859] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:11.641204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:11.644245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:11.747999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06d723-9eed-4cc3-a38b-85ba65481a5c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:11.753133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82327869-984e-4432-8431-8623db6d9ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:11.753974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82327869-984e-4432-8431-8623db6d9ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:11.754873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82327869-984e-4432-8431-8623db6d9ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:11.755747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82327869-984e-4432-8431-8623db6d9ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:11.767985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82327869-984e-4432-8431-8623db6d9ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:11.769685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82327869-984e-4432-8431-8623db6d9ad6 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:11.799233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 938/1860] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:11.799618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70303a30-fa8f-4b02-b857-f8cc99e055de admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:11.818031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:11.818994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:11.819530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:11.819952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] PATH_INFO: `/v3/projects/5f8bb1dfaf2849d493c1f157057fcb64` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:11.822756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8bb1dfaf2849d493c1f157057fcb64)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:11.828334 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:11.985002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:12.065606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8bb1dfaf2849d493c1f157057fcb64', 'request_id': 'req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.089847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8bb1dfaf2849d493c1f157057fcb64', 'request_id': 'req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.106360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8bb1dfaf2849d493c1f157057fcb64', 'request_id': 'req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.176495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] The token cache is being invalidate because project 5f8bb1dfaf2849d493c1f157057fcb64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:12.176495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.177228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c1722e24-9ff8-4a2c-8add-9e8f832c0fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.179271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 923/1861] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:11 2022] DELETE /identity/v3/projects/5f8bb1dfaf2849d493c1f157057fcb64 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:12.188849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098453c7-cf65-40e5-a4b9-3b7d5f8f5349 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:12.189334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098453c7-cf65-40e5-a4b9-3b7d5f8f5349 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:12.190030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098453c7-cf65-40e5-a4b9-3b7d5f8f5349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:12.217744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:12.252279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-098453c7-cf65-40e5-a4b9-3b7d5f8f5349 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:12.254374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-098453c7-cf65-40e5-a4b9-3b7d5f8f5349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:12.399548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 939/1862] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:12.401028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d775dd-d6ec-40ce-bada-a8735322b78a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:12.401699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d775dd-d6ec-40ce-bada-a8735322b78a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:12.402135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d775dd-d6ec-40ce-bada-a8735322b78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:12.449519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e5d775dd-d6ec-40ce-bada-a8735322b78a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:12.450897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e5d775dd-d6ec-40ce-bada-a8735322b78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:12.482673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098453c7-cf65-40e5-a4b9-3b7d5f8f5349 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:12.483941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 924/1863] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:12.500603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098453c7-cf65-40e5-a4b9-3b7d5f8f5349 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:12.507085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:12.508192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:12.508364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:12.508697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:12.509978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:12.509978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:12.530687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:12.576700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d775dd-d6ec-40ce-bada-a8735322b78a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:12.577878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 940/1864] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:12.594968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d775dd-d6ec-40ce-bada-a8735322b78a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:12.604279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:12.605461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:12.605956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:12.607307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:12.608416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:12.615134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:12.686911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:12.704675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 925/1865] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:12 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:12.718141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3685e-5f30-4d47-ad16-042f97e4b1d1 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:12.718245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:12.719206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:12.825487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:12.826456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:12.826745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:12.827047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:12.827909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:12.831456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:12.848013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed7287-d38c-4d78-a8bf-27e917e8c5dc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:12.855407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:12.856699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:12.856699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:12.862635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] PATH_INFO: `/v3/users/5e0fa1a5cb4645e4977fea4f34265c8c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:12.863956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0fa1a5cb4645e4977fea4f34265c8c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:12.866009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:12.871490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 941/1866] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:12 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:12.873526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:12.875987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:12.892251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff21d3e-e4d6-44cd-9c3b-fdadaf078020 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:12.898736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:12.899561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:12.899916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:12.900301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] PATH_INFO: `/v3/projects/7fc73cc5c64b428cb1ddc98fbc406ff8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:12.902364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc73cc5c64b428cb1ddc98fbc406ff8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:12.904039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:12.967594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0fa1a5cb4645e4977fea4f34265c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.968257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0fa1a5cb4645e4977fea4f34265c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.968375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc73cc5c64b428cb1ddc98fbc406ff8', 'request_id': 'req-0ca1c188-f717-4af5-8a06-c400c6608bae'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.977205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0fa1a5cb4645e4977fea4f34265c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.981423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc73cc5c64b428cb1ddc98fbc406ff8', 'request_id': 'req-0ca1c188-f717-4af5-8a06-c400c6608bae'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.991449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0fa1a5cb4645e4977fea4f34265c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:12.995113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc73cc5c64b428cb1ddc98fbc406ff8', 'request_id': 'req-0ca1c188-f717-4af5-8a06-c400c6608bae'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.010706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] The token cache is being invalidate because project 7fc73cc5c64b428cb1ddc98fbc406ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:13.010987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.011798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.013349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 942/1867] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:12 2022] DELETE /identity/v3/projects/7fc73cc5c64b428cb1ddc98fbc406ff8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:13.014251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:13.015821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:13.018445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0fa1a5cb4645e4977fea4f34265c8c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.022522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:13.040124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 926/1868] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:12 2022] DELETE /identity/v3/users/5e0fa1a5cb4645e4977fea4f34265c8c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:13.040124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:13.041385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:13.059307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:13.116209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1c188-f717-4af5-8a06-c400c6608bae admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:13.196090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e632d4-a235-4d67-84b1-945212b60e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:13.197719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e632d4-a235-4d67-84b1-945212b60e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:13.198206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e632d4-a235-4d67-84b1-945212b60e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:13.198643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e632d4-a235-4d67-84b1-945212b60e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:13.199471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e632d4-a235-4d67-84b1-945212b60e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:13.201756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e632d4-a235-4d67-84b1-945212b60e20 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:13.222488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 943/1869] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:13.224214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5351c-6286-4782-b557-84433bd6c248 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:13.224509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5351c-6286-4782-b557-84433bd6c248 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:13.224786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5351c-6286-4782-b557-84433bd6c248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:13.237562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdf1fd-9ba5-4116-8558-c76d42ed9a5a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:13.259613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:13.263353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:13.263353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:13.265864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] PATH_INFO: `/v3/users/ef92911721ae454bbf372bc519a87e8b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:13.269062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef92911721ae454bbf372bc519a87e8b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:13.276247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:13.281529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ffe5351c-6286-4782-b557-84433bd6c248 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:13.282962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ffe5351c-6286-4782-b557-84433bd6c248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:13.303873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 944/1870] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:13.304836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5351c-6286-4782-b557-84433bd6c248 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:13.326112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5351c-6286-4782-b557-84433bd6c248 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:13.332006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:13.332981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:13.333269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:13.333906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:13.334735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:13.336762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:13.349129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 945/1871] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:13.365983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:13.365983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:13.424219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef92911721ae454bbf372bc519a87e8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.427206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef92911721ae454bbf372bc519a87e8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.457841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef92911721ae454bbf372bc519a87e8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.483422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef92911721ae454bbf372bc519a87e8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.551953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef92911721ae454bbf372bc519a87e8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:13.565040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f947c-ecf3-40af-8298-1b5633f80586 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:13.583016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d814d-e9a9-40fa-b03e-1eeb1fc99dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:13.583982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d814d-e9a9-40fa-b03e-1eeb1fc99dcb service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:13.584471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d814d-e9a9-40fa-b03e-1eeb1fc99dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:13.584911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d814d-e9a9-40fa-b03e-1eeb1fc99dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:13.595533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d32d814d-e9a9-40fa-b03e-1eeb1fc99dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:13.601688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 927/1872] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:13 2022] DELETE /identity/v3/users/ef92911721ae454bbf372bc519a87e8b => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:13.603258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:13.616705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:13.623132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d814d-e9a9-40fa-b03e-1eeb1fc99dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:13.625470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d814d-e9a9-40fa-b03e-1eeb1fc99dcb service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:13.643196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 946/1873] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:13.749035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7c96-2ed1-4a8b-bca0-34aeafa69c0b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:13.749035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:13.749035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:13.749035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:13.749719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] PATH_INFO: `/v3/projects/6ac906fef7a84442a13fd8488c484dd3/users/cfbf103c180541dc9078cb1e00e872c3/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:13.792266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac906fef7a84442a13fd8488c484dd3, user_id=cfbf103c180541dc9078cb1e00e872c3, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:13.797094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:13.819343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 928/1874] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:13 2022] PUT /identity/v3/projects/6ac906fef7a84442a13fd8488c484dd3/users/cfbf103c180541dc9078cb1e00e872c3/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:13.831868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:13.832142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:13.832421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:13.900633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] MFA Rules not processed for user `cfbf103c180541dc9078cb1e00e872c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:13.904582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:13.938955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:14.024133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 947/1875] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:14.038569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:14.043219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:14.053688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:14.109867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:14.110903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:14.132483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864796c7-b1c2-4699-bd64-022fb9a4ef4e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:14.138281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:14.139934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:14.139934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:14.139934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:14.156049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:14.158512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:14.176552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 929/1876] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:14.206910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:14.270661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d8c6c-3a38-484c-9cc2-5de591cb932e None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:14.277988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:14.278914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:14.279193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:14.279466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] PATH_INFO: `/v3/users/312e4f8bc2164c4bbca79e5186726594` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:14.285222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=312e4f8bc2164c4bbca79e5186726594)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:14.288688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:14.313349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9871-fc2c-4052-858d-6472ff70683a service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:14.325598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:14.326850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:14.327479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:14.328429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] PATH_INFO: `/v3/users/cfbf103c180541dc9078cb1e00e872c3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:14.331664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbf103c180541dc9078cb1e00e872c3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:14.338172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:14.472789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312e4f8bc2164c4bbca79e5186726594'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.487899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312e4f8bc2164c4bbca79e5186726594'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.493319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312e4f8bc2164c4bbca79e5186726594'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.509229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312e4f8bc2164c4bbca79e5186726594'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.511779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbf103c180541dc9078cb1e00e872c3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.515196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbf103c180541dc9078cb1e00e872c3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.527160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbf103c180541dc9078cb1e00e872c3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.543545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbf103c180541dc9078cb1e00e872c3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.546082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a09581bd-9ea3-49a2-94ce-138b52ffa1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312e4f8bc2164c4bbca79e5186726594'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.566425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d9f61c-d4b9-4574-9b7d-202a799f6b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbf103c180541dc9078cb1e00e872c3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:14.571270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 948/1877] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:14 2022] DELETE /identity/v3/users/312e4f8bc2164c4bbca79e5186726594 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:14.573575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:14.573998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:14.574400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:14.602144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 930/1878] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:14 2022] DELETE /identity/v3/users/cfbf103c180541dc9078cb1e00e872c3 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:14.610160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:14.610710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:14.611144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:14.627952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:14.629462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:14.649320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:14.651753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:14.678708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:14.723819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:14.763303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 931/1879] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:14.764023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 949/1880] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:14.828044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:14.830511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:14.836795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:14.838980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:14.992636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595ae50-b2c3-4aad-bfdc-3fc1b596c6a3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:14.998229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b29a6a-bb54-4e40-8dae-e2af4cbdd158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:15.002949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b29a6a-bb54-4e40-8dae-e2af4cbdd158 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:15.003835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b29a6a-bb54-4e40-8dae-e2af4cbdd158 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:15.004692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b29a6a-bb54-4e40-8dae-e2af4cbdd158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:15.022000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ff41b-5b32-4b2e-b605-0c502bfd4cbe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:15.030303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:15.034197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:15.034197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:15.034197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:15.058759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:15.061397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:15.092093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 932/1881] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:15.093136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b29a6a-bb54-4e40-8dae-e2af4cbdd158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:15.098575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b29a6a-bb54-4e40-8dae-e2af4cbdd158 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:15.130388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 950/1882] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:15.421346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:15.424952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:15.469427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fadc6-61c9-41b5-a7e8-1c78de81d42c service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:15.495485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:15.498664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:15.499657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:15.500734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] PATH_INFO: `/v3/projects/7b3696abdf1943de8c88cb5c45834fdd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:15.503699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3696abdf1943de8c88cb5c45834fdd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:15.511234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:15.631425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3696abdf1943de8c88cb5c45834fdd', 'request_id': 'req-06763a1c-de62-41e3-aaae-6ead6952cc67'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:15.641425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3696abdf1943de8c88cb5c45834fdd', 'request_id': 'req-06763a1c-de62-41e3-aaae-6ead6952cc67'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:15.648898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3696abdf1943de8c88cb5c45834fdd', 'request_id': 'req-06763a1c-de62-41e3-aaae-6ead6952cc67'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:15.675112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] The token cache is being invalidate because project 7b3696abdf1943de8c88cb5c45834fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:15.675797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:15.676962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:15.679821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 933/1883] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:15 2022] DELETE /identity/v3/projects/7b3696abdf1943de8c88cb5c45834fdd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:15.685784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f080-3017-477d-8602-9c83505e6041 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:15.686674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f080-3017-477d-8602-9c83505e6041 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:15.687098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f080-3017-477d-8602-9c83505e6041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:15.722401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3777f080-3017-477d-8602-9c83505e6041 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:15.724169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3777f080-3017-477d-8602-9c83505e6041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:15.804917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:15.807199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:15.838988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 951/1884] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:15.840819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d99e-2fbc-4f00-84fb-0d99bd7705f7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:15.841187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d99e-2fbc-4f00-84fb-0d99bd7705f7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:15.841616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d99e-2fbc-4f00-84fb-0d99bd7705f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:15.872796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3659d99e-2fbc-4f00-84fb-0d99bd7705f7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:15.875629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3659d99e-2fbc-4f00-84fb-0d99bd7705f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:15.886217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 952/1885] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:15.892307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d99e-2fbc-4f00-84fb-0d99bd7705f7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:15.906452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d99e-2fbc-4f00-84fb-0d99bd7705f7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:15.917407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:15.918807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:15.919106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:15.919385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:15.925953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:15.925953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:15.925953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:15.952237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 953/1886] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:15.953511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:15.975911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:15.977982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c83f7-b50b-4dc0-a589-137e8ca6dd5a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:15.988859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:15.989911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:15.990493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:15.990988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:15.991864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:15.994389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.014958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06763a1c-de62-41e3-aaae-6ead6952cc67 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.022492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.023170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.023454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.023841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] PATH_INFO: `/v3/projects/dddeef41a10e4939a923e806076aee3a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.025096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dddeef41a10e4939a923e806076aee3a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.029153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.036982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 954/1887] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:15 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:16.042371 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.113879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:16.124001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dddeef41a10e4939a923e806076aee3a', 'request_id': 'req-85858a76-45de-432a-ab6e-cd7df0461801'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.149558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1aff8-caf4-48f0-b962-17463a66d7d9 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.158737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dddeef41a10e4939a923e806076aee3a', 'request_id': 'req-85858a76-45de-432a-ab6e-cd7df0461801'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.170477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dddeef41a10e4939a923e806076aee3a', 'request_id': 'req-85858a76-45de-432a-ab6e-cd7df0461801'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.208455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] The token cache is being invalidate because project dddeef41a10e4939a923e806076aee3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:16.209011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.217922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.225264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 934/1888] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:15 2022] DELETE /identity/v3/projects/dddeef41a10e4939a923e806076aee3a => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:16.229256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.231156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:16.262037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.262994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.263564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.263831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.264856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.277986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.312786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 955/1889] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:16 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:16.316928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.319466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:16.336206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908aa4-ad66-4ad8-8889-80cf0a87099a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.342513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.343147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.343491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.344376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.345266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.347025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.357354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85858a76-45de-432a-ab6e-cd7df0461801 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.362446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 956/1890] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:16.366911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.368947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.369914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.370500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.370701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] PATH_INFO: `/v3/users/df0d947771d9419c98a2e42bb0a28df4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.372183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0d947771d9419c98a2e42bb0a28df4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.375641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.381718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b941-b072-49e5-9c24-d39acf74b730 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.390854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.391786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.392317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.392802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.393747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.396907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.441608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 957/1891] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:16.442831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.460187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f61b-1eb5-48ff-9644-d419135cd501 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.475273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.475952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0d947771d9419c98a2e42bb0a28df4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.476657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.476783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0d947771d9419c98a2e42bb0a28df4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.477294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.477829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] PATH_INFO: `/v3/projects/21c45bced482403d873eb87eaa2a2c12/users/5687bed1403042f497f7573205470fe1/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.494258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0d947771d9419c98a2e42bb0a28df4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.511287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0d947771d9419c98a2e42bb0a28df4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.521644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c45bced482403d873eb87eaa2a2c12, user_id=5687bed1403042f497f7573205470fe1, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.532897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.539230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ecdc4694-0cc7-4da2-8d3a-129100ca22ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0d947771d9419c98a2e42bb0a28df4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:16.551574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 935/1892] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:16 2022] DELETE /identity/v3/users/df0d947771d9419c98a2e42bb0a28df4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:16.569843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 958/1893] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:16 2022] PUT /identity/v3/projects/21c45bced482403d873eb87eaa2a2c12/users/5687bed1403042f497f7573205470fe1/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:16.587345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bba03f-b236-4bce-9a1b-966c3bd042ff None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.587345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bba03f-b236-4bce-9a1b-966c3bd042ff None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.587345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bba03f-b236-4bce-9a1b-966c3bd042ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.596891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:16.643032 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-38bba03f-b236-4bce-9a1b-966c3bd042ff None None] MFA Rules not processed for user `5687bed1403042f497f7573205470fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:16.644286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-38bba03f-b236-4bce-9a1b-966c3bd042ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:16.756958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 936/1894] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:16.768449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.771220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:16.882269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce669b-e732-402a-aee7-f241e2fe8377 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.888193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e48c6-ed59-4137-a42b-39ffde0f3020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.889537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e48c6-ed59-4137-a42b-39ffde0f3020 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.890120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e48c6-ed59-4137-a42b-39ffde0f3020 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.891513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e48c6-ed59-4137-a42b-39ffde0f3020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.905467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e48c6-ed59-4137-a42b-39ffde0f3020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.908036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e48c6-ed59-4137-a42b-39ffde0f3020 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.923636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 959/1895] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:16.926834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a30cb5-25d9-4edd-bdc1-c66edcc1291d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.927256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a30cb5-25d9-4edd-bdc1-c66edcc1291d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.927537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a30cb5-25d9-4edd-bdc1-c66edcc1291d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.931564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 937/1896] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:58:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:58:16.949646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.950104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.950410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:17.003490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:17.004766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.061517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:17.162327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 960/1897] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:58:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:17.233812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a30cb5-25d9-4edd-bdc1-c66edcc1291d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:17.235763 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b3a30cb5-25d9-4edd-bdc1-c66edcc1291d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.391515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a30cb5-25d9-4edd-bdc1-c66edcc1291d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:17.403189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:17.403442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:17.403678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:17.403951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] PATH_INFO: `/v3/projects/6ac906fef7a84442a13fd8488c484dd3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:17.409030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac906fef7a84442a13fd8488c484dd3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:17.411244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:17.493809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:17.496095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.530104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0646cf2-a607-4928-bf76-936c51517f18 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:17.536618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac906fef7a84442a13fd8488c484dd3', 'request_id': 'req-0f17c909-3ced-41ef-91f2-61ae136aeb14'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.542809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac906fef7a84442a13fd8488c484dd3', 'request_id': 'req-0f17c909-3ced-41ef-91f2-61ae136aeb14'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.563144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac906fef7a84442a13fd8488c484dd3', 'request_id': 'req-0f17c909-3ced-41ef-91f2-61ae136aeb14'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.582994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] The token cache is being invalidate because project 6ac906fef7a84442a13fd8488c484dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:17.582994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.585274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f17c909-3ced-41ef-91f2-61ae136aeb14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.588130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 938/1898] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:17 2022] DELETE /identity/v3/projects/6ac906fef7a84442a13fd8488c484dd3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:17.594131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:17.594422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:17.594713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:17.646095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:17.647415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.663373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:17.667332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:17.667887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:17.668316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] PATH_INFO: `/v3/projects/2bbb869d1a69445f8622d3b7941f4b8e` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:17.669912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bbb869d1a69445f8622d3b7941f4b8e)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:17.672816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:17.677698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:17.764599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 939/1899] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:17.765811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bbb869d1a69445f8622d3b7941f4b8e', 'request_id': 'req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.765910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:17.767400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.772144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bbb869d1a69445f8622d3b7941f4b8e', 'request_id': 'req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.786062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bbb869d1a69445f8622d3b7941f4b8e', 'request_id': 'req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.801268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] The token cache is being invalidate because project 2bbb869d1a69445f8622d3b7941f4b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:17.801636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.803044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8d8ab1b4-f0b6-4116-b010-de6e79c9cccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:17.808499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 961/1900] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:17 2022] DELETE /identity/v3/projects/2bbb869d1a69445f8622d3b7941f4b8e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:17.810211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:17.810729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:17.811187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:17.838555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:17.840630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.879799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea41566-07a4-4796-b80e-aca3fa564877 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:17.938239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:17.955366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 962/1901] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:17.955366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:17.957065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.991129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:17.991867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:17.992110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47529-75c4-43c7-87ee-3598fd7662a1 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:17.992218 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:17.992683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:17.997968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:18.003991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.004748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.006059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.006059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] PATH_INFO: `/v3/users/086cd4da730f4e77b7a4161f5719ae40` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.006832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=086cd4da730f4e77b7a4161f5719ae40)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.009658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.042257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.050378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.084016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 940/1902] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:18.088021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.107090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228e053-4188-4487-bf12-aacb91d6fb22 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.112236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.113629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.113999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.114282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.115079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:18.115681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.115992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086cd4da730f4e77b7a4161f5719ae40'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.119037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.132917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086cd4da730f4e77b7a4161f5719ae40'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.135128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086cd4da730f4e77b7a4161f5719ae40'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.142698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 941/1903] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:18.143597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.145483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:18.165315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086cd4da730f4e77b7a4161f5719ae40'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.181389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39c72b-69a4-4549-9653-1c424a7dc8ca admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.190753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.191366 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.191651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.192033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] PATH_INFO: `/v3/projects/35151454e49a4f4eb31507575b46325b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.194690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35151454e49a4f4eb31507575b46325b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.196808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.213285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086cd4da730f4e77b7a4161f5719ae40'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.231249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 963/1904] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:17 2022] DELETE /identity/v3/users/086cd4da730f4e77b7a4161f5719ae40 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:18.235288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.238856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:18.257046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:18.301199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35151454e49a4f4eb31507575b46325b', 'request_id': 'req-6b83a496-f248-49ec-addf-2464200af40d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.313163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35151454e49a4f4eb31507575b46325b', 'request_id': 'req-6b83a496-f248-49ec-addf-2464200af40d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.325563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35151454e49a4f4eb31507575b46325b', 'request_id': 'req-6b83a496-f248-49ec-addf-2464200af40d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.347251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] The token cache is being invalidate because project 35151454e49a4f4eb31507575b46325b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:18.347808 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.348677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6b83a496-f248-49ec-addf-2464200af40d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:18.350856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 942/1905] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:18 2022] DELETE /identity/v3/projects/35151454e49a4f4eb31507575b46325b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:18.357241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9910-4047-4ab0-9a9a-6364eeba2501 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.357886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9910-4047-4ab0-9a9a-6364eeba2501 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.358360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9910-4047-4ab0-9a9a-6364eeba2501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.371786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9c620-7c58-4f11-a0e2-7c34b25bab38 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.394577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c80f9910-4047-4ab0-9a9a-6364eeba2501 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:18.399125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c80f9910-4047-4ab0-9a9a-6364eeba2501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:18.457207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.458329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.458605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.458871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.459510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.461495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.499878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 964/1906] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:18 2022] POST /identity/v3/projects => generated 377 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:18.504017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.505789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 943/1907] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:18.506470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:18.514356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:18.518186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e0af-6299-49c0-b816-c0937c844b26 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.518477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e0af-6299-49c0-b816-c0937c844b26 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.520252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e0af-6299-49c0-b816-c0937c844b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.545103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a576e0af-6299-49c0-b816-c0937c844b26 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:18.547294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a576e0af-6299-49c0-b816-c0937c844b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:18.633545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf244534-de59-46c8-b661-82bf3905ac6c admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.641004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.641643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.641989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.642268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.643215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.645343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.653827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 944/1908] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:18.655703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576e0af-6299-49c0-b816-c0937c844b26 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.667038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576e0af-6299-49c0-b816-c0937c844b26 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.672348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.672971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.673663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.673981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.674642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:18.674950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.676725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.679137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 965/1909] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:18 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:58:18.683521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.694840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 945/1910] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:18.697379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dae526-4dbb-4798-9864-a0f878028ff6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.706326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.707695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.709129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.709673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.710717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.712861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.725879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.730886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 966/1911] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:18.731704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.744459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022175e-a267-4d06-89c7-c2dc688189df admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.745034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854a812-bcea-4a79-adea-b5059f170555 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.751546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.752177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.752455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.752730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.753042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.753468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.755167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.756139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.756578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.757048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.767944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.773749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 967/1912] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:18.775466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.775716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.796679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63771-12e4-4260-8630-a47ae9f90b86 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.826899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:18.827597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:18.833272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:18.834293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] PATH_INFO: `/v3/projects/dd1f626d07ce4c84a8edbf5eb218e753/users/18297969a96a499da21f807217a3528b/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:18.850004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 946/1913] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:18 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:18.850990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.854634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:18.860271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1f626d07ce4c84a8edbf5eb218e753, user_id=18297969a96a499da21f807217a3528b, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:18.865368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:18.890181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 968/1914] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:18 2022] PUT /identity/v3/projects/dd1f626d07ce4c84a8edbf5eb218e753/users/18297969a96a499da21f807217a3528b/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:18.890181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:18.905221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:18.972611 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358a0a4-b4d2-4c49-81a2-a0d860c7102c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:18.996180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806e6b0-952b-4858-a653-eb910865e753 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.004336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808ad13-39e8-4793-9e36-229e21f6c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.005274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ad13-39e8-4793-9e36-229e21f6c5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.005655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ad13-39e8-4793-9e36-229e21f6c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.006065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808ad13-39e8-4793-9e36-229e21f6c5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.006869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808ad13-39e8-4793-9e36-229e21f6c5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.009322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808ad13-39e8-4793-9e36-229e21f6c5a1 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.059672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 969/1915] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:18 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:19.064934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03cd7-83c0-43b8-897d-2b89a7284283 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.066324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03cd7-83c0-43b8-897d-2b89a7284283 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.068109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03cd7-83c0-43b8-897d-2b89a7284283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.092719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.093942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.094233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.094512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] PATH_INFO: `/v3/projects/4310b33484fc4e0aa04304a682e95f5e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.098701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4310b33484fc4e0aa04304a682e95f5e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.100866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.149833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-30d03cd7-83c0-43b8-897d-2b89a7284283 None None] MFA Rules not processed for user `18297969a96a499da21f807217a3528b`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:19.151928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-30d03cd7-83c0-43b8-897d-2b89a7284283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:19.189807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:19.203834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4310b33484fc4e0aa04304a682e95f5e', 'request_id': 'req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:19.215392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4310b33484fc4e0aa04304a682e95f5e', 'request_id': 'req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:19.226878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4310b33484fc4e0aa04304a682e95f5e', 'request_id': 'req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:19.246871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] The token cache is being invalidate because project 4310b33484fc4e0aa04304a682e95f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:19.247791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:19.248570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:19.250704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 947/1916] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:18 2022] DELETE /identity/v3/projects/4310b33484fc4e0aa04304a682e95f5e => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:19.255841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.258267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:19.282434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 970/1917] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:19.284362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.284695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.284963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.309113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:19.310776 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:19.373215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272bc3f-a1c6-4ea1-ae4c-e3f781fdbee1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.379972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c2006-dde2-41a2-8707-66763ea4ff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.380608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2006-dde2-41a2-8707-66763ea4ff72 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.380890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2006-dde2-41a2-8707-66763ea4ff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.381426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2006-dde2-41a2-8707-66763ea4ff72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.382129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c2006-dde2-41a2-8707-66763ea4ff72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.383743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c2006-dde2-41a2-8707-66763ea4ff72 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.385427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:19.388193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 971/1918] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:19.389133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.391253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:19.404423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 948/1919] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:19.406841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de716f-47ce-42db-bd35-fdd422cb60ba None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.407688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de716f-47ce-42db-bd35-fdd422cb60ba None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.407878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de716f-47ce-42db-bd35-fdd422cb60ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.437500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d5de716f-47ce-42db-bd35-fdd422cb60ba None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:19.438955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d5de716f-47ce-42db-bd35-fdd422cb60ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:19.448131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 949/1920] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:19.449294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de716f-47ce-42db-bd35-fdd422cb60ba None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.461070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de716f-47ce-42db-bd35-fdd422cb60ba None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.467666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.469234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.469696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.470179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.471011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.472949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.484678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 950/1921] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:19.485906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.496843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3633673-2b6c-48c3-a0c6-1cdcf2feff29 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.503061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.504037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.504509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.504926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.505831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:19.506142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.507771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.525157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 951/1922] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:19.525157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.533311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08e42a-6bdf-48b1-8d4d-86cac7502abc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.538355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3709a8-795f-4ffe-b63e-f52ab8bba750 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.538982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.539895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.540234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.540477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] PATH_INFO: `/v3/projects/528ed7fd606c4d3db5614007ee8b2854/users/125a10c2c21844e2b5c63b0b41bac294/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.543398 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb7aa-b09c-41dc-ac5f-b8e42f062dd6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a24bc4a0c4f480192d87a27a243acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.544068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb7aa-b09c-41dc-ac5f-b8e42f062dd6 service glance] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.544401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb7aa-b09c-41dc-ac5f-b8e42f062dd6 service glance] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.544684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb7aa-b09c-41dc-ac5f-b8e42f062dd6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.546916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c5dcb7aa-b09c-41dc-ac5f-b8e42f062dd6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:19.561900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb7aa-b09c-41dc-ac5f-b8e42f062dd6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.563188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528ed7fd606c4d3db5614007ee8b2854, user_id=125a10c2c21844e2b5c63b0b41bac294, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.563887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb7aa-b09c-41dc-ac5f-b8e42f062dd6 service glance] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.566242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.584891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 952/1923] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:19 2022] PUT /identity/v3/projects/528ed7fd606c4d3db5614007ee8b2854/users/125a10c2c21844e2b5c63b0b41bac294/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:19.585749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.598314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 972/1924] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:19.599474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d54b-e357-406a-8c87-643c8039c245 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.599764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d54b-e357-406a-8c87-643c8039c245 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.600044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d54b-e357-406a-8c87-643c8039c245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.647967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-2506d54b-e357-406a-8c87-643c8039c245 None None] MFA Rules not processed for user `125a10c2c21844e2b5c63b0b41bac294`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:19.649252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2506d54b-e357-406a-8c87-643c8039c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:19.689080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d567718-00bd-4d5e-8166-ea76d2c1a038 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.694084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.694710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.694989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.695248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.695880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.697960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.721454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:19.731117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 953/1925] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:19.736657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.787393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 973/1926] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:19.788669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506d54b-e357-406a-8c87-643c8039c245 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.814002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506d54b-e357-406a-8c87-643c8039c245 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.835251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6d261-003c-425c-a743-996058bf4cf8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.855070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.855724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.856013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.856285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.857087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.861571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.902295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 954/1927] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/users => generated 282 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:19.905387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.907857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.908554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.908842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.909123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.909879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.912087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.918032 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcff945-29dc-4a2f-a0c5-3581806534d8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:19.928989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:19.930399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:19.930939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:19.931530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:19.932774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:19.935750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:19.954700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 955/1928] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:19.957206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 974/1929] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:19.959517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.963952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:19.967850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:19.973235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a3381-e8dd-470a-9b71-b5c95d278d55 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.065405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.066522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.066665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.066883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.067676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.069480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.075680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd54d6d-11dc-4c36-bdd4-5e1a4e0a951e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.082406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.082813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 956/1930] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:20.083292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.083633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.083907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.084550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.086516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.088443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.105376 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fad165-13c8-40fd-bb03-1ab63d7e354d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.113106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 975/1931] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:19 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:20.118997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.121163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c44fc6-d5e8-4970-a3c0-a251514b4a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.122176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44fc6-d5e8-4970-a3c0-a251514b4a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.122674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44fc6-d5e8-4970-a3c0-a251514b4a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.123199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44fc6-d5e8-4970-a3c0-a251514b4a41 admin admin] PATH_INFO: `/v3/projects/fdef3fe4e2d7410285dffebf8250f742/users/533e9b76bf604728a2090a2e3c741713/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.136675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced59e0-a8b0-4535-b3a0-58b832268964 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.152896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.153604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.153956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.154375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.155075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.156856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.161112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c44fc6-d5e8-4970-a3c0-a251514b4a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdef3fe4e2d7410285dffebf8250f742, user_id=533e9b76bf604728a2090a2e3c741713, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.164070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c44fc6-d5e8-4970-a3c0-a251514b4a41 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.169125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 976/1932] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:20.177049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.182463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 957/1933] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:20 2022] PUT /identity/v3/projects/fdef3fe4e2d7410285dffebf8250f742/users/533e9b76bf604728a2090a2e3c741713/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:20.193371 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f15d9-8dc9-40be-b506-d45d204d6c61 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.195291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530559-eb14-471c-8cc9-39858e32aef6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.195650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530559-eb14-471c-8cc9-39858e32aef6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.195863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530559-eb14-471c-8cc9-39858e32aef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.233293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-93530559-eb14-471c-8cc9-39858e32aef6 None None] MFA Rules not processed for user `533e9b76bf604728a2090a2e3c741713`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:20.235160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-93530559-eb14-471c-8cc9-39858e32aef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:20.243666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:20.299347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.300120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.300473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.300831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] PATH_INFO: `/v3/projects/849c9503fda341409e6aa128e64eb61b/users/e8b396f532914114892421e1337d81b5/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.341834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849c9503fda341409e6aa128e64eb61b, user_id=e8b396f532914114892421e1337d81b5, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.352033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.382705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 977/1934] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:20 2022] PUT /identity/v3/projects/849c9503fda341409e6aa128e64eb61b/users/e8b396f532914114892421e1337d81b5/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:20.386970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.398028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 958/1935] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:20.402606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530559-eb14-471c-8cc9-39858e32aef6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.430172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530559-eb14-471c-8cc9-39858e32aef6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.516074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc91c6-2065-4011-8480-758026c296a4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.522962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.523601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.523902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.524310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.525104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.527203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.527332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.528480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.529051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.529862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.531868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.535163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.542161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:20.563220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 978/1936] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:20.565462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.571985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 959/1937] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:20 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:20.575987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.580093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b24fe4-c785-4232-b072-56ea536594d2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.650391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a1900-d755-4f39-8dfa-24b4100eb9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.650879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a1900-d755-4f39-8dfa-24b4100eb9bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.651157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a1900-d755-4f39-8dfa-24b4100eb9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.651438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a1900-d755-4f39-8dfa-24b4100eb9bb admin admin] PATH_INFO: `/v3/projects/849c9503fda341409e6aa128e64eb61b/users/e8b396f532914114892421e1337d81b5/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.653845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a1900-d755-4f39-8dfa-24b4100eb9bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=849c9503fda341409e6aa128e64eb61b, user_id=e8b396f532914114892421e1337d81b5, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.655609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7ec3a-baed-4bea-945f-2047931676fc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.656519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a1900-d755-4f39-8dfa-24b4100eb9bb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.660527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.661098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.661381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.661646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.662454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.664385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.667751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 979/1938] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:20 2022] PUT /identity/v3/projects/849c9503fda341409e6aa128e64eb61b/users/e8b396f532914114892421e1337d81b5/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:20.679969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.680384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.680520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.695719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 960/1939] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:20 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:58:20.696827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.707027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5280-7d1e-4ccd-9eaa-885a5a7ab65e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.711650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] MFA Rules not processed for user `e8b396f532914114892421e1337d81b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:20.712876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:20.762902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:20.767224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.767903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.768149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.768396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.769056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.770730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.791035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 961/1940] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:20.792929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 980/1941] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:20.795894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.807400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.810314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbe17-9625-4131-a4bc-03c4e4619d34 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.810959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:20.816477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.817173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.817451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.817723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] PATH_INFO: `/v3/projects/fbf9cc8ac81044c7a2b677a911cd044f/users/acc209794f384c17839d35ab50392a74/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.838961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf9cc8ac81044c7a2b677a911cd044f, user_id=acc209794f384c17839d35ab50392a74, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:20.843647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:20.872034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 962/1942] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:20 2022] PUT /identity/v3/projects/fbf9cc8ac81044c7a2b677a911cd044f/users/acc209794f384c17839d35ab50392a74/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:20.874090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:20.918305 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918cd0e-fcaa-4ec8-81ff-a5c831216d9c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:20.979619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:20.988516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:20.989788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:20.990314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:20.991457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:20.996177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d89e5-1323-48b0-ad76-01a473fea700 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:21.003124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:21.004436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:21.005035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:21.006029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:21.008401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:21.010084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:21.022262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 963/1943] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:21.026288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:21.037706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e665-7a10-4f91-a69a-791b24bd5cb1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:21.043568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6d8d4-c0f4-4846-9315-416d5cdcc01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:21.044118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6d8d4-c0f4-4846-9315-416d5cdcc01f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:21.044397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6d8d4-c0f4-4846-9315-416d5cdcc01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:21.044669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6d8d4-c0f4-4846-9315-416d5cdcc01f admin admin] PATH_INFO: `/v3/projects/fbf9cc8ac81044c7a2b677a911cd044f/users/acc209794f384c17839d35ab50392a74/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:21.046927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6d8d4-c0f4-4846-9315-416d5cdcc01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf9cc8ac81044c7a2b677a911cd044f, user_id=acc209794f384c17839d35ab50392a74, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:21.049906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6d8d4-c0f4-4846-9315-416d5cdcc01f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:21.066283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 964/1944] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:21 2022] PUT /identity/v3/projects/fbf9cc8ac81044c7a2b677a911cd044f/users/acc209794f384c17839d35ab50392a74/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:21.078412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68127a-a559-4f45-a61b-9e2277f0f67b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:21.078717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68127a-a559-4f45-a61b-9e2277f0f67b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:21.079141 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68127a-a559-4f45-a61b-9e2277f0f67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:21.079941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:21.081872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:21.106319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 981/1945] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:21.116224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fb68127a-a559-4f45-a61b-9e2277f0f67b None None] MFA Rules not processed for user `acc209794f384c17839d35ab50392a74`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:21.117599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb68127a-a559-4f45-a61b-9e2277f0f67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:21.224508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 965/1946] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:21.239213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:21.249042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f16d4-25c4-4964-8d72-88788b2a673c service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:21.299014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:21.304667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:21.305294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:21.305574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:21.305902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:21.320846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:21.322826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:21.335023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 982/1947] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:23.036979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68127a-a559-4f45-a61b-9e2277f0f67b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:23.139588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68127a-a559-4f45-a61b-9e2277f0f67b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:23.146965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:23.160548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ce54d-0800-4215-a4dd-096d8d0d1dd1 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:23.163911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:23.164577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:23.165438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:23.165438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] PATH_INFO: `/v3/users/125a10c2c21844e2b5c63b0b41bac294` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:23.167096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=125a10c2c21844e2b5c63b0b41bac294)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:23.168722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:23.174662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:23.175356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:23.175635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:23.175905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] PATH_INFO: `/v3/users/533e9b76bf604728a2090a2e3c741713` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:23.178151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=533e9b76bf604728a2090a2e3c741713)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:23.181893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:23.250087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125a10c2c21844e2b5c63b0b41bac294'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.251464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125a10c2c21844e2b5c63b0b41bac294'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.263523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125a10c2c21844e2b5c63b0b41bac294'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.267193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '533e9b76bf604728a2090a2e3c741713'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.277099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125a10c2c21844e2b5c63b0b41bac294'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.279893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:23.285590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '533e9b76bf604728a2090a2e3c741713'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.287423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '533e9b76bf604728a2090a2e3c741713'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.297946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cef7ff5f-20ef-4fe8-a566-a8f9d39ca4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125a10c2c21844e2b5c63b0b41bac294'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.301161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '533e9b76bf604728a2090a2e3c741713'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.307240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 966/1948] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:23 2022] DELETE /identity/v3/users/125a10c2c21844e2b5c63b0b41bac294 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:23.317816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:23.317816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:23.317816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:23.332317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d37fc59-824c-4e62-b00f-df5365b8ce5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '533e9b76bf604728a2090a2e3c741713'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:23.337974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:23.339575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:23.340391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 983/1949] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:23 2022] DELETE /identity/v3/users/533e9b76bf604728a2090a2e3c741713 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:23.344988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:23.345283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:23.345550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:23.375816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:23.377044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:23.438672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 967/1950] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:23.449388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:23.450949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:23.491970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 984/1951] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:23.502846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:23.509298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:23.582659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d441c6-fe26-4500-a660-329936bc8b35 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:23.594848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:23.595500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:23.595802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:23.596057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:23.610076 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:23.612468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:23.642521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 968/1952] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:23.692144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7715d57-a865-4060-93fe-7f4fa9887978 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:23.704462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:23.705141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:23.705976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:23.705976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:23.735519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:23.741824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:23.771792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 985/1953] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:58:24.284953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:24.287684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:24.301601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324296d-a87c-4b1f-8b3f-22f76495a1e1 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:24.304717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:24.305524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:24.305655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:24.305992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] PATH_INFO: `/v3/projects/fdef3fe4e2d7410285dffebf8250f742` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:24.307182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdef3fe4e2d7410285dffebf8250f742)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:24.309107 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:24.369997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdef3fe4e2d7410285dffebf8250f742', 'request_id': 'req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.377443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdef3fe4e2d7410285dffebf8250f742', 'request_id': 'req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.384424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdef3fe4e2d7410285dffebf8250f742', 'request_id': 'req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.400311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] The token cache is being invalidate because project fdef3fe4e2d7410285dffebf8250f742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:24.400718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.401552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.403570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 969/1954] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:24 2022] DELETE /identity/v3/projects/fdef3fe4e2d7410285dffebf8250f742 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:24.408017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:24.417895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb808-c220-4873-9f95-cd18a7ab283f service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:24.422868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:24.496916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:24.498398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:24.511572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:24.512573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:24.513074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:24.513219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:24.513497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] PATH_INFO: `/v3/users/acc209794f384c17839d35ab50392a74` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:24.514762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc209794f384c17839d35ab50392a74)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:24.518159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:24.614773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee289bfa-2828-40b5-8a8a-cbdbc47ac324 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:24.621139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:24.621694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:24.622069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:24.622346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] PATH_INFO: `/v3/projects/528ed7fd606c4d3db5614007ee8b2854` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:24.623658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=528ed7fd606c4d3db5614007ee8b2854)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:24.625797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:24.625975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc209794f384c17839d35ab50392a74'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.639600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc209794f384c17839d35ab50392a74'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.642798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc209794f384c17839d35ab50392a74'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.656782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc209794f384c17839d35ab50392a74'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.678164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc209794f384c17839d35ab50392a74'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.692100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 986/1955] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:24 2022] DELETE /identity/v3/users/acc209794f384c17839d35ab50392a74 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:24.706684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528ed7fd606c4d3db5614007ee8b2854', 'request_id': 'req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.714670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528ed7fd606c4d3db5614007ee8b2854', 'request_id': 'req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.722199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528ed7fd606c4d3db5614007ee8b2854', 'request_id': 'req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.743181 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] The token cache is being invalidate because project 528ed7fd606c4d3db5614007ee8b2854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:24.743818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.744982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:24.747958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 970/1956] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:24 2022] DELETE /identity/v3/projects/528ed7fd606c4d3db5614007ee8b2854 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:24.752921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:24.754520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:24.845436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf77b-b42c-4217-b50d-bc309aa61fd8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:24.853677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:24.854885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:24.854885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:24.855097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] PATH_INFO: `/v3/users/e8b396f532914114892421e1337d81b5` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:24.857116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b396f532914114892421e1337d81b5)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:24.861505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:24.954773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:25.002383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b396f532914114892421e1337d81b5'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.022542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b396f532914114892421e1337d81b5'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.024170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b396f532914114892421e1337d81b5'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.056186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b396f532914114892421e1337d81b5'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.113877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b396f532914114892421e1337d81b5'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.139285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 987/1957] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:24 2022] DELETE /identity/v3/users/e8b396f532914114892421e1337d81b5 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:25.238842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:25.240389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:25.371816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ac30-9bc5-4bc9-9fb7-4c378c98b085 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:25.495650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:25.583255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:25.583922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:25.584204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:25.584486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] PATH_INFO: `/v3/projects/fbf9cc8ac81044c7a2b677a911cd044f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:25.591929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf9cc8ac81044c7a2b677a911cd044f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:25.603220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:25.710876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf9cc8ac81044c7a2b677a911cd044f', 'request_id': 'req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.731820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf9cc8ac81044c7a2b677a911cd044f', 'request_id': 'req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.747940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf9cc8ac81044c7a2b677a911cd044f', 'request_id': 'req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.818316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:25.820010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:25.835757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] The token cache is being invalidate because project fbf9cc8ac81044c7a2b677a911cd044f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:25.837072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.841667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-af30bfd4-97d6-428a-adbc-7ac5f6394f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:25.847707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 971/1958] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:25 2022] DELETE /identity/v3/projects/fbf9cc8ac81044c7a2b677a911cd044f => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:25.856139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a8ab6-9a5b-4b90-8ce8-fddc2a512bf4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:25.856793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274ab04-1a24-4843-8419-71280eead528 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:25.856793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274ab04-1a24-4843-8419-71280eead528 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:25.856793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274ab04-1a24-4843-8419-71280eead528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:25.910691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d274ab04-1a24-4843-8419-71280eead528 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:25.911952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d274ab04-1a24-4843-8419-71280eead528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:25.984603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:25.985021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:25.985303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:25.985590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] PATH_INFO: `/v3/projects/849c9503fda341409e6aa128e64eb61b` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:25.987228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=849c9503fda341409e6aa128e64eb61b)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:25.989296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.012651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 972/1959] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:26.021100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.021484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.021910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.047642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:26.049391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.072553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849c9503fda341409e6aa128e64eb61b', 'request_id': 'req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.089834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849c9503fda341409e6aa128e64eb61b', 'request_id': 'req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.118564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849c9503fda341409e6aa128e64eb61b', 'request_id': 'req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.144087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] The token cache is being invalidate because project 849c9503fda341409e6aa128e64eb61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:26.144396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.145145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c8d6a41e-06e0-4a2c-ae1f-e061a4b198fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.148177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 988/1960] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:25 2022] DELETE /identity/v3/projects/849c9503fda341409e6aa128e64eb61b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:26.153482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dae81-acfc-4824-a8fe-5a73c49b51c7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.153798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dae81-acfc-4824-a8fe-5a73c49b51c7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.154092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dae81-acfc-4824-a8fe-5a73c49b51c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.196190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 973/1961] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:26.197327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.198960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-873dae81-acfc-4824-a8fe-5a73c49b51c7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:26.199979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.200251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-873dae81-acfc-4824-a8fe-5a73c49b51c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.221923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 989/1962] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:26.224588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dae81-acfc-4824-a8fe-5a73c49b51c7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.238957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dae81-acfc-4824-a8fe-5a73c49b51c7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.245978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951a031-b2d8-412b-b9a9-0e5536c958e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.246614 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951a031-b2d8-412b-b9a9-0e5536c958e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.246956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951a031-b2d8-412b-b9a9-0e5536c958e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.247238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951a031-b2d8-412b-b9a9-0e5536c958e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.248135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951a031-b2d8-412b-b9a9-0e5536c958e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:26.248449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951a031-b2d8-412b-b9a9-0e5536c958e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.250944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951a031-b2d8-412b-b9a9-0e5536c958e7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.259964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c9316-2e8e-4343-8f31-e42d99a39c96 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.273125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 990/1963] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:26.277093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa02e7-b7f9-487f-959a-493e4a7a31e7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.278519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa02e7-b7f9-487f-959a-493e4a7a31e7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.279055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa02e7-b7f9-487f-959a-493e4a7a31e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.285492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.287398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.287733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.288130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] PATH_INFO: `/v3/users/5687bed1403042f497f7573205470fe1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.294650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5687bed1403042f497f7573205470fe1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.303452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.310746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5cfa02e7-b7f9-487f-959a-493e4a7a31e7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:26.323557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5cfa02e7-b7f9-487f-959a-493e4a7a31e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.341702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 991/1964] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:26.345805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:26.347691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa02e7-b7f9-487f-959a-493e4a7a31e7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.362068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa02e7-b7f9-487f-959a-493e4a7a31e7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.371026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.372152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.372672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.372948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.373755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.376168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.383465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:26.420042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 992/1965] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:26.421318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.428334 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5687bed1403042f497f7573205470fe1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.429003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5687bed1403042f497f7573205470fe1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.439551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5687bed1403042f497f7573205470fe1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.452196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5687bed1403042f497f7573205470fe1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.495083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5687bed1403042f497f7573205470fe1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:26.508412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 974/1966] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:26 2022] DELETE /identity/v3/users/5687bed1403042f497f7573205470fe1 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:26.509288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.511157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.564974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87c97c-3c0e-450f-9552-bf6caae6873d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.698351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3721710-8589-43ea-97df-d6c931aafd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.699048 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721710-8589-43ea-97df-d6c931aafd43 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.699461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721710-8589-43ea-97df-d6c931aafd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.699936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721710-8589-43ea-97df-d6c931aafd43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.700688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3721710-8589-43ea-97df-d6c931aafd43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:26.701062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3721710-8589-43ea-97df-d6c931aafd43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.703303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3721710-8589-43ea-97df-d6c931aafd43 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.708782 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33dec5-95e0-4eae-9b21-4e039f47fe99 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.716190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.717509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.717509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.717756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.718392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.720419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.722301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 993/1967] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:26.723655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5b07b-a374-4ecb-8af1-e059607eb349 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.724003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5b07b-a374-4ecb-8af1-e059607eb349 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.724355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5b07b-a374-4ecb-8af1-e059607eb349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.750243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 975/1968] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:26.750243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.750581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.753273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-05f5b07b-a374-4ecb-8af1-e059607eb349 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:26.754074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-05f5b07b-a374-4ecb-8af1-e059607eb349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.763306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ae341-090e-464f-9d4e-d5ab8ea88cbe admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.766243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 994/1969] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:26.767392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5b07b-a374-4ecb-8af1-e059607eb349 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.772463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.773153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.775061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.775061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.775061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.776604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.779805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5b07b-a374-4ecb-8af1-e059607eb349 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.785196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.785950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.786304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.786629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.787346 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.789179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.803986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 995/1970] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:26.804872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.806681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:26.817825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 976/1971] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:26.818917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.891046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac695afa-624d-4401-a660-8014369a0f32 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.908616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:26.916935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03348685-ea9c-4223-a9e7-7c40080eef75 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.921405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:26.928424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.929226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.929820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.930209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.932045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.934207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.953094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 977/1972] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:26.953094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:26.964290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.965391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.965789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.965889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78027168-b624-43f5-a720-6d21ce1f304a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:26.966314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.971748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:26.972539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:26.972830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:26.973109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:26.973790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.975941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:26.981251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:26.984871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.005414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 996/1973] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:27.008811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 978/1974] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:26 2022] POST /identity/v3/users => generated 292 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:27.009617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.015671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.021650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a954c-ca37-476e-a4a2-03ea5ea39d62 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.026877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75883ab8-ba21-4e04-8a70-54c5122bf1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.029666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e58105-97d0-49d8-afe5-91fcf41df979 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.029900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883ab8-ba21-4e04-8a70-54c5122bf1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.030075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883ab8-ba21-4e04-8a70-54c5122bf1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.030336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883ab8-ba21-4e04-8a70-54c5122bf1cf admin admin] PATH_INFO: `/v3/projects/3379b4d9719d4c66a277818f728942fe/users/a4fb223600154d618d2313a2d88a40aa/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.037921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.039027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.039521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.039999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.041435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.043802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.057752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75883ab8-ba21-4e04-8a70-54c5122bf1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3379b4d9719d4c66a277818f728942fe, user_id=a4fb223600154d618d2313a2d88a40aa, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.062257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 979/1975] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:27.063152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75883ab8-ba21-4e04-8a70-54c5122bf1cf admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.074227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.099730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 997/1976] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:27 2022] PUT /identity/v3/projects/3379b4d9719d4c66a277818f728942fe/users/a4fb223600154d618d2313a2d88a40aa/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:27.109347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b3a94-0280-464d-b2ae-e8d6c42c3fef admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.113516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890141e-1009-4935-bd3a-4de58ab1664b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.113913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890141e-1009-4935-bd3a-4de58ab1664b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.114203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890141e-1009-4935-bd3a-4de58ab1664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.152983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e890141e-1009-4935-bd3a-4de58ab1664b None None] MFA Rules not processed for user `a4fb223600154d618d2313a2d88a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:27.154865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e890141e-1009-4935-bd3a-4de58ab1664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:27.268147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.268799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.269084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.269424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.270135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.272246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.297931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 980/1977] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:27.299616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.307443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 998/1978] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:27.308448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890141e-1009-4935-bd3a-4de58ab1664b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.315252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87e7d-8714-4c76-9340-f88a8cc5c88b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.323290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.323708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890141e-1009-4935-bd3a-4de58ab1664b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.324285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.324778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.325128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] PATH_INFO: `/v3/projects/c7ef1cf23ff9476a8ca691a9cbd43e45/users/5bff408d47e244c5837c49d0098f3fc0/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.331019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1ee0e-c9e3-445d-a2f7-027cae030007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.331670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ee0e-c9e3-445d-a2f7-027cae030007 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.331960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ee0e-c9e3-445d-a2f7-027cae030007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.332230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ee0e-c9e3-445d-a2f7-027cae030007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.332903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1ee0e-c9e3-445d-a2f7-027cae030007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.335074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1ee0e-c9e3-445d-a2f7-027cae030007 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.354146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ef1cf23ff9476a8ca691a9cbd43e45, user_id=5bff408d47e244c5837c49d0098f3fc0, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.365537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.382387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 981/1979] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:27 2022] PUT /identity/v3/projects/c7ef1cf23ff9476a8ca691a9cbd43e45/users/5bff408d47e244c5837c49d0098f3fc0/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:27.385948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 999/1980] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:27 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:27.390780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.394622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.394933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.395310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.453688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] MFA Rules not processed for user `5bff408d47e244c5837c49d0098f3fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:27.455032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:27.505883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a560443-cbb9-497c-b82b-23d616d245b3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.589063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:27.614745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1000/1981] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:27.614745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.632376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:27.639551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.640824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.642468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.643085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.645275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.658727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.667941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:27.678956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6b721-39b4-4aa6-86b3-e9ea79359e9c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.687988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.688839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.689239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.689615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] PATH_INFO: `/v3/projects/21c45bced482403d873eb87eaa2a2c12` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.691101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c45bced482403d873eb87eaa2a2c12)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.694204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.723950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 982/1982] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:27 2022] POST /identity/v3/users => generated 294 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:27.723950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.750987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3699d4d-374a-41d9-a3c6-f21a18e50637 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.757135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.757972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.758357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.758736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.762186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.766160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.811634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 983/1983] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:27 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:27.815275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.823438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c45bced482403d873eb87eaa2a2c12', 'request_id': 'req-f07caa68-7417-470c-9f83-41d427392d32'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:27.846534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c45bced482403d873eb87eaa2a2c12', 'request_id': 'req-f07caa68-7417-470c-9f83-41d427392d32'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:27.857942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c45bced482403d873eb87eaa2a2c12', 'request_id': 'req-f07caa68-7417-470c-9f83-41d427392d32'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:27.882888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] The token cache is being invalidate because project 21c45bced482403d873eb87eaa2a2c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:27.883522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:27.885025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:27.888752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1001/1984] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:27 2022] DELETE /identity/v3/projects/21c45bced482403d873eb87eaa2a2c12 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:27.907457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b91bbf-5d66-45cd-8fd8-a1685a650438 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:27.914596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:27.916872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:27.929073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba09e30-4ab9-49be-b41e-bac0ad9b70a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:27.929731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba09e30-4ab9-49be-b41e-bac0ad9b70a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.930112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba09e30-4ab9-49be-b41e-bac0ad9b70a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.930632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba09e30-4ab9-49be-b41e-bac0ad9b70a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:27.931116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba09e30-4ab9-49be-b41e-bac0ad9b70a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:27.932886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba09e30-4ab9-49be-b41e-bac0ad9b70a5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:27.952295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 984/1985] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:27.953636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:27.953960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:27.954501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.006419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:28.008364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:28.034347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 985/1986] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:28.034347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.034347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:28.042649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07caa68-7417-470c-9f83-41d427392d32 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.042893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add9347-1849-4d73-8b8b-610376440c0f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.049098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb62eec-d4a8-4ad4-94b3-37ce62bd6f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.049281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.049812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.050110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.050386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] PATH_INFO: `/v3/projects/2d959f285cff48d0968a6a3f709f6f98/users/8d7947b8ded748e59b5d98c35a5f1c99/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.050488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb62eec-d4a8-4ad4-94b3-37ce62bd6f90 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.050836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb62eec-d4a8-4ad4-94b3-37ce62bd6f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.051168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb62eec-d4a8-4ad4-94b3-37ce62bd6f90 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.051956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb62eec-d4a8-4ad4-94b3-37ce62bd6f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.055163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb62eec-d4a8-4ad4-94b3-37ce62bd6f90 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.070026 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d959f285cff48d0968a6a3f709f6f98, user_id=8d7947b8ded748e59b5d98c35a5f1c99, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.080841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1002/1987] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:27 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:28.082420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165596b-ff3b-494f-b2f6-86a6da9225b4 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.082908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165596b-ff3b-494f-b2f6-86a6da9225b4 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.083338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165596b-ff3b-494f-b2f6-86a6da9225b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.087940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.115872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 986/1988] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:28 2022] PUT /identity/v3/projects/2d959f285cff48d0968a6a3f709f6f98/users/8d7947b8ded748e59b5d98c35a5f1c99/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:28.117350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.124293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c165596b-ff3b-494f-b2f6-86a6da9225b4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:28.125808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c165596b-ff3b-494f-b2f6-86a6da9225b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:28.224586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1003/1989] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:28.226657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165596b-ff3b-494f-b2f6-86a6da9225b4 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.241274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165596b-ff3b-494f-b2f6-86a6da9225b4 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.247337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.248679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.249161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.250219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.251119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.253917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.261218 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6527b7-ec89-467c-b9f2-f9351520df27 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.267280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.268901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.269056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1004/1990] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:28.269466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.269924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.271122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.271801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.274504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.277731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:28.283034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0479c-4ce5-4d6f-ba10-07101b9e9d84 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.291634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 987/1991] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:28.292362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.306008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.306649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.306929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.307494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.307860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:28.308169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.308418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994ad77-94a6-41bc-981d-3426532da3d8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.310147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.314570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.315461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.315695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.316056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] PATH_INFO: `/v3/projects/2d959f285cff48d0968a6a3f709f6f98/users/8d7947b8ded748e59b5d98c35a5f1c99/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.319200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d959f285cff48d0968a6a3f709f6f98, user_id=8d7947b8ded748e59b5d98c35a5f1c99, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.322112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.327043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1005/1992] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:28.328813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.336552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:28.344094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb0f8f-f941-4073-97fb-38f158cf0927 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.349884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 988/1993] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:28 2022] PUT /identity/v3/projects/2d959f285cff48d0968a6a3f709f6f98/users/8d7947b8ded748e59b5d98c35a5f1c99/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:28.351965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.440868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a13dd-ff9a-46cb-92cf-1cc575663310 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.442906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.443645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.444032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.444321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] PATH_INFO: `/v3/projects/b2c76aea4c264521b618c85cc0ef8e86/users/377338522e9e4313a69d75a3162dbbf6/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.447631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c82ece-c182-48c5-9393-537a7b681cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.448197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82ece-c182-48c5-9393-537a7b681cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.448489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82ece-c182-48c5-9393-537a7b681cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.448851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82ece-c182-48c5-9393-537a7b681cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.449441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c82ece-c182-48c5-9393-537a7b681cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.452648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c82ece-c182-48c5-9393-537a7b681cdb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.477377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c76aea4c264521b618c85cc0ef8e86, user_id=377338522e9e4313a69d75a3162dbbf6, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.480946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.500436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 989/1994] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:28 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:28.503910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2d2c6-c607-41db-8a2b-0c149d375937 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.504507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2d2c6-c607-41db-8a2b-0c149d375937 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.505582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2d2c6-c607-41db-8a2b-0c149d375937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.510643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1006/1995] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:28 2022] PUT /identity/v3/projects/b2c76aea4c264521b618c85cc0ef8e86/users/377338522e9e4313a69d75a3162dbbf6/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:28.513663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.563740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e3e2d2c6-c607-41db-8a2b-0c149d375937 None None] MFA Rules not processed for user `8d7947b8ded748e59b5d98c35a5f1c99`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:28.566049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e3e2d2c6-c607-41db-8a2b-0c149d375937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:28.632074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8633f-d7c8-43bb-a394-1229d727c16f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.637731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.638566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.638857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.639219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.641007 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.642995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.687772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1007/1996] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:28 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:28.688917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.711890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ae79c-9144-4d7e-976b-eec9b0b928f4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.724386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 990/1997] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:28.730196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2d2c6-c607-41db-8a2b-0c149d375937 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.734429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.735074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.736945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.737247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.737964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.746932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.758407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2d2c6-c607-41db-8a2b-0c149d375937 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.774740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.775442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.775758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.776037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.777109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1008/1998] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:28.778661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.778773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.782278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.783167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:28.832870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 991/1999] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:28.834132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.847637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ce75f-6a27-4d02-bd19-4c6d1a2fb746 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.852817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:28.853520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:28.853838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:28.854085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] PATH_INFO: `/v3/projects/b2c76aea4c264521b618c85cc0ef8e86/users/377338522e9e4313a69d75a3162dbbf6/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:28.856168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c76aea4c264521b618c85cc0ef8e86, user_id=377338522e9e4313a69d75a3162dbbf6, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:28.859528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:28.875286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 992/2000] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:28 2022] PUT /identity/v3/projects/b2c76aea4c264521b618c85cc0ef8e86/users/377338522e9e4313a69d75a3162dbbf6/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:28.886273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:28.907608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de8b7-8de5-4db6-a5ec-f7fd5e398938 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:28.916107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:28.976763 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:29.021114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ac1bd-76bb-4a0d-b9b3-359f3b66d9e1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:29.029469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49f73a-72d4-477c-8da4-3c3d7ea8cc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:29.030195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f73a-72d4-477c-8da4-3c3d7ea8cc28 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.031650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f73a-72d4-477c-8da4-3c3d7ea8cc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.031946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f73a-72d4-477c-8da4-3c3d7ea8cc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.032068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:29.032709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49f73a-72d4-477c-8da4-3c3d7ea8cc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:29.032823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.033008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.033195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.034816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49f73a-72d4-477c-8da4-3c3d7ea8cc28 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:29.035345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:29.047883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 993/2001] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:29.060261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5b66-41bf-4fbc-977f-088e710991e1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.060699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5b66-41bf-4fbc-977f-088e710991e1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.060986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5b66-41bf-4fbc-977f-088e710991e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.109088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-964e5b66-41bf-4fbc-977f-088e710991e1 None None] MFA Rules not processed for user `377338522e9e4313a69d75a3162dbbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:29.110375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-964e5b66-41bf-4fbc-977f-088e710991e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:29.229785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:29.231562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:29.251646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 994/2002] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:29.257469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e5b66-41bf-4fbc-977f-088e710991e1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:29.269507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e5b66-41bf-4fbc-977f-088e710991e1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:29.285414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1009/2003] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:29.299654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a4467-6c6a-40bc-bad3-234aa01abc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:29.301868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4467-6c6a-40bc-bad3-234aa01abc9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.302234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4467-6c6a-40bc-bad3-234aa01abc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.303658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4467-6c6a-40bc-bad3-234aa01abc9e admin admin] PATH_INFO: `/v3/projects/2bedf9e0fe0f49eda1bf8fe707b22ca3/users/cbcdfa261595481cb8c739857fa6b60a/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.362128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4467-6c6a-40bc-bad3-234aa01abc9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bedf9e0fe0f49eda1bf8fe707b22ca3, user_id=cbcdfa261595481cb8c739857fa6b60a, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:29.365156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4467-6c6a-40bc-bad3-234aa01abc9e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:29.374440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:29.376013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:29.395650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11295a72-73ae-47d1-94c2-361148f483f8 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:29.397029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 995/2004] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:29 2022] PUT /identity/v3/projects/2bedf9e0fe0f49eda1bf8fe707b22ca3/users/cbcdfa261595481cb8c739857fa6b60a/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:29.407525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fafdf2-843c-4c11-b31f-d26982e8c2b0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.407813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fafdf2-843c-4c11-b31f-d26982e8c2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.408092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fafdf2-843c-4c11-b31f-d26982e8c2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.468447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-92fafdf2-843c-4c11-b31f-d26982e8c2b0 None None] MFA Rules not processed for user `cbcdfa261595481cb8c739857fa6b60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:29.472954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-92fafdf2-843c-4c11-b31f-d26982e8c2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:29.515637 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:29.516256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.516530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.516803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.632501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 996/2005] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:29.648976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fafdf2-843c-4c11-b31f-d26982e8c2b0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:29.660854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fafdf2-843c-4c11-b31f-d26982e8c2b0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:29.665392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:29.666308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.666786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.667203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.678494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:29.680446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:29.683843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:29.687811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:29.707803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 997/2006] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:29.714638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1010/2007] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:29.960461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:29.970825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac69f3-90a4-4a31-a63e-c0ae212d3de4 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:29.975411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab771fa-c430-4c84-ba8d-544cef53d732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:29.976385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab771fa-c430-4c84-ba8d-544cef53d732 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:29.976724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab771fa-c430-4c84-ba8d-544cef53d732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:29.976919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab771fa-c430-4c84-ba8d-544cef53d732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:29.979548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dab771fa-c430-4c84-ba8d-544cef53d732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:30.002787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:30.122434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab771fa-c430-4c84-ba8d-544cef53d732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:30.131193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab771fa-c430-4c84-ba8d-544cef53d732 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:30.151775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 998/2008] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:30.549227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:30.564646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:30.615276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6104584-1012-4591-abdb-98ecaa24afa3 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:30.620417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:30.621108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:30.621335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:30.621606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] PATH_INFO: `/v3/users/5bff408d47e244c5837c49d0098f3fc0` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:30.622821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bff408d47e244c5837c49d0098f3fc0)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:30.624833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:30.674627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bff408d47e244c5837c49d0098f3fc0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:30.682453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bff408d47e244c5837c49d0098f3fc0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:30.683219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bff408d47e244c5837c49d0098f3fc0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:30.690904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bff408d47e244c5837c49d0098f3fc0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:30.703844 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bff408d47e244c5837c49d0098f3fc0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:30.709746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1011/2009] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:30 2022] DELETE /identity/v3/users/5bff408d47e244c5837c49d0098f3fc0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:30.714653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:30.715145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:30.715250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:30.734619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:30.735830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:30.806870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 999/2010] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:30.817820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:30.819447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:30.878649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74cfd-0310-47ad-ae9e-a1aa5c9868fc admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:30.882877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:30.883508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:30.883787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:30.884093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:30.896568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:30.898051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:30.910166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1012/2011] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:31.507657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:31.511924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:31.533735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d0f44-d126-4931-838a-f821387e6daf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:31.544602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:31.545436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:31.545875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:31.546108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] PATH_INFO: `/v3/projects/c7ef1cf23ff9476a8ca691a9cbd43e45` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:31.548663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ef1cf23ff9476a8ca691a9cbd43e45)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:31.550645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:31.663294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ef1cf23ff9476a8ca691a9cbd43e45', 'request_id': 'req-4ace92d7-89ba-462f-809e-cb1596f1395d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.671765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ef1cf23ff9476a8ca691a9cbd43e45', 'request_id': 'req-4ace92d7-89ba-462f-809e-cb1596f1395d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.678601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ef1cf23ff9476a8ca691a9cbd43e45', 'request_id': 'req-4ace92d7-89ba-462f-809e-cb1596f1395d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.693446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] The token cache is being invalidate because project c7ef1cf23ff9476a8ca691a9cbd43e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:31.694113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.694909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.697846 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1000/2012] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:31 2022] DELETE /identity/v3/projects/c7ef1cf23ff9476a8ca691a9cbd43e45 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:31.700932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:31.714755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ebdc5-2891-4d92-8f8a-8974ac380945 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:31.772950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:31.806267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:31.807037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:31.807387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:31.807756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] PATH_INFO: `/v3/users/377338522e9e4313a69d75a3162dbbf6` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:31.809136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377338522e9e4313a69d75a3162dbbf6)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:31.813106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:31.895586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377338522e9e4313a69d75a3162dbbf6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.906670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377338522e9e4313a69d75a3162dbbf6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.907469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377338522e9e4313a69d75a3162dbbf6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.916357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377338522e9e4313a69d75a3162dbbf6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.939705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-06737c83-1014-4287-800c-664b6cdce0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377338522e9e4313a69d75a3162dbbf6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:31.950886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1013/2013] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:31 2022] DELETE /identity/v3/users/377338522e9e4313a69d75a3162dbbf6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:32.880146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:32.881630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:32.901197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace92d7-89ba-462f-809e-cb1596f1395d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:32.946670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:32.969051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:32.969960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:32.970250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:32.970530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] PATH_INFO: `/v3/projects/b2c76aea4c264521b618c85cc0ef8e86` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:32.971712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c76aea4c264521b618c85cc0ef8e86)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:32.973652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.026859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c76aea4c264521b618c85cc0ef8e86', 'request_id': 'req-9845ab49-982d-4e1f-b1f8-62d6d282abf4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:33.035304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c76aea4c264521b618c85cc0ef8e86', 'request_id': 'req-9845ab49-982d-4e1f-b1f8-62d6d282abf4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:33.041387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c76aea4c264521b618c85cc0ef8e86', 'request_id': 'req-9845ab49-982d-4e1f-b1f8-62d6d282abf4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:33.054328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] The token cache is being invalidate because project b2c76aea4c264521b618c85cc0ef8e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:33.054621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:33.055285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9845ab49-982d-4e1f-b1f8-62d6d282abf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:33.056783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1001/2014] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:32 2022] DELETE /identity/v3/projects/b2c76aea4c264521b618c85cc0ef8e86 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:33.062362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f60b93-5de8-4a6e-be00-e4599ce0cc1d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.062824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f60b93-5de8-4a6e-be00-e4599ce0cc1d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.062913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f60b93-5de8-4a6e-be00-e4599ce0cc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.082854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-16f60b93-5de8-4a6e-be00-e4599ce0cc1d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:33.084093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-16f60b93-5de8-4a6e-be00-e4599ce0cc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:33.155159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1014/2015] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:33.157369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c4ccc-6096-4460-ad3d-80e499545161 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.157655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c4ccc-6096-4460-ad3d-80e499545161 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.157991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c4ccc-6096-4460-ad3d-80e499545161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.179185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-379c4ccc-6096-4460-ad3d-80e499545161 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:33.180546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-379c4ccc-6096-4460-ad3d-80e499545161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:33.195415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1002/2016] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:33.199497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f60b93-5de8-4a6e-be00-e4599ce0cc1d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:33.209484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f60b93-5de8-4a6e-be00-e4599ce0cc1d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:33.215929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:33.216542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.216812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.217538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.217716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:33.218080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:33.219638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.243565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1015/2017] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:33.247870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c4ccc-6096-4460-ad3d-80e499545161 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:33.257888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c4ccc-6096-4460-ad3d-80e499545161 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:33.262988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:33.263691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.263973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.264247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.265429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:33.267353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.302288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1003/2018] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:33 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:33.303945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:33.320222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4664039-8078-4503-bf86-1c6ba9c9e3bc admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:33.375149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:33.404214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:33.404921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.405298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.405733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.406484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:33.408500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.438208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1016/2019] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:33 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:33.439513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:33.455074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad29e6f-4197-4ea8-85c7-f2419dd31bdf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:33.460342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:33.461060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.461557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.461950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.462822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:33.464847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.481787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1004/2020] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:33.485604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:33.497788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c825e6-f14f-4b5e-aed7-b97ccbd78012 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:33.503931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acaaa2-ae87-4c24-b713-9690259ea7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:33.504535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acaaa2-ae87-4c24-b713-9690259ea7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.504816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acaaa2-ae87-4c24-b713-9690259ea7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.505087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acaaa2-ae87-4c24-b713-9690259ea7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.506340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acaaa2-ae87-4c24-b713-9690259ea7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:33.510777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acaaa2-ae87-4c24-b713-9690259ea7b6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.525843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1017/2021] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:33.527522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:33.538046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2e82e-7797-4612-a1de-7bdff4ad5f01 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:33.548446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:33.549119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.549401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.549667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] PATH_INFO: `/v3/projects/cacc64a1dc88491c90a4e3552c85a482/users/892ba3dad26b4ce6ae17ec425acc5087/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.565291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cacc64a1dc88491c90a4e3552c85a482, user_id=892ba3dad26b4ce6ae17ec425acc5087, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:33.568069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.581291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1005/2022] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:33 2022] PUT /identity/v3/projects/cacc64a1dc88491c90a4e3552c85a482/users/892ba3dad26b4ce6ae17ec425acc5087/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:33.595206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.595488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.595809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.634017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] MFA Rules not processed for user `892ba3dad26b4ce6ae17ec425acc5087`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:33.635285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:33.714515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1018/2023] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:33.726101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:33.727975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:33.794008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a624f34-342a-4d60-a360-8a80f1cf2a34 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:33.800459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:33.801450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:33.802067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:33.802359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:33.812405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:33.813870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:33.834464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1006/2024] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:35.337380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:35.340422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:35.403012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:35.425519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63d8f5-1f05-4204-bd3c-f2540bdaefa8 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:35.434610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3345a-0ef6-40d3-aea5-6667b1808fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:35.436472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3345a-0ef6-40d3-aea5-6667b1808fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:35.437506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3345a-0ef6-40d3-aea5-6667b1808fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:35.438021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3345a-0ef6-40d3-aea5-6667b1808fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:35.444445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40e3345a-0ef6-40d3-aea5-6667b1808fb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:35.467551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c067c-6bbe-4aed-b6a5-6d1d67dd3644 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:35.468209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:35.474080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:35.474733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:35.475011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:35.475286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] PATH_INFO: `/v3/users/cbcdfa261595481cb8c739857fa6b60a` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:35.478561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbcdfa261595481cb8c739857fa6b60a)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:35.480921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:35.538538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbcdfa261595481cb8c739857fa6b60a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:35.544550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:35.546633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3345a-0ef6-40d3-aea5-6667b1808fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:35.548284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3345a-0ef6-40d3-aea5-6667b1808fb7 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:35.549973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbcdfa261595481cb8c739857fa6b60a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:35.554181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbcdfa261595481cb8c739857fa6b60a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:35.566442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbcdfa261595481cb8c739857fa6b60a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:35.569381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1007/2025] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:35.604119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbcdfa261595481cb8c739857fa6b60a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:35.616227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1019/2026] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:35 2022] DELETE /identity/v3/users/cbcdfa261595481cb8c739857fa6b60a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:35.621077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:35.621620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:35.621982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:35.648606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:35.649944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:35.730318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1008/2027] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:35.739777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:35.742193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:35.825004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d649e33-73f9-4847-a202-d02f9ad71ad5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:35.832517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fa757-d4a9-4f84-b876-3994486de813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:35.833242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fa757-d4a9-4f84-b876-3994486de813 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:35.833588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fa757-d4a9-4f84-b876-3994486de813 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:35.834117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fa757-d4a9-4f84-b876-3994486de813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:35.848006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fa757-d4a9-4f84-b876-3994486de813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:35.849459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fa757-d4a9-4f84-b876-3994486de813 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:35.860696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1020/2028] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:36.152818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.154286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:36.166554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9326c-d811-4efb-b050-da13b97830e8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.171883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.172541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.172804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.173207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] PATH_INFO: `/v3/projects/2bedf9e0fe0f49eda1bf8fe707b22ca3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.174695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bedf9e0fe0f49eda1bf8fe707b22ca3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.176583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.225286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bedf9e0fe0f49eda1bf8fe707b22ca3', 'request_id': 'req-5ea06034-8f4a-4d46-9dd0-680984bf2111'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:36.232806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bedf9e0fe0f49eda1bf8fe707b22ca3', 'request_id': 'req-5ea06034-8f4a-4d46-9dd0-680984bf2111'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:36.240684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bedf9e0fe0f49eda1bf8fe707b22ca3', 'request_id': 'req-5ea06034-8f4a-4d46-9dd0-680984bf2111'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:36.255020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] The token cache is being invalidate because project 2bedf9e0fe0f49eda1bf8fe707b22ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:36.255312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:36.257951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ea06034-8f4a-4d46-9dd0-680984bf2111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:36.260148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1009/2029] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:36 2022] DELETE /identity/v3/projects/2bedf9e0fe0f49eda1bf8fe707b22ca3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:36.265219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbed8a-65c9-4030-935c-0b4046c2f535 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.265570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbed8a-65c9-4030-935c-0b4046c2f535 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.265950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbed8a-65c9-4030-935c-0b4046c2f535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.287266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8ccbed8a-65c9-4030-935c-0b4046c2f535 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:36.288470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8ccbed8a-65c9-4030-935c-0b4046c2f535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:36.351442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1021/2030] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:36.360933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f93c86-bb33-4280-869b-b1037a04cce9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.361217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f93c86-bb33-4280-869b-b1037a04cce9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.361491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f93c86-bb33-4280-869b-b1037a04cce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.383831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d9f93c86-bb33-4280-869b-b1037a04cce9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:36.385045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9f93c86-bb33-4280-869b-b1037a04cce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:36.392970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1010/2031] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:36.397220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbed8a-65c9-4030-935c-0b4046c2f535 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.407102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbed8a-65c9-4030-935c-0b4046c2f535 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.411584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.412175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.412447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.412713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.413423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:36.413651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.415241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.427068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1022/2032] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:58:36.431474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f93c86-bb33-4280-869b-b1037a04cce9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.440462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f93c86-bb33-4280-869b-b1037a04cce9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.445081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.445673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.445972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.446261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.446864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.448727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.472109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:36.484051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1011/2033] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:36 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:36.485173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.496190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c047-c5fc-4568-a6ec-422807f55176 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.556334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.556948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.557234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.557500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.558184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.560092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.568180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:36.583772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1023/2034] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:36 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:36.588159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.600069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c3dc5-aca3-4cb0-9a77-9d4794529b56 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.605969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.606587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.606865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.607135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.607784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.609508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.623373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1012/2035] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:36.625576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.635458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72208c8-b11e-406b-b89d-5ec9607bd532 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.640775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282cab7-8340-4ec4-bb21-c9bf57a1b2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.641446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282cab7-8340-4ec4-bb21-c9bf57a1b2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.641731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282cab7-8340-4ec4-bb21-c9bf57a1b2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.642038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282cab7-8340-4ec4-bb21-c9bf57a1b2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.642804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282cab7-8340-4ec4-bb21-c9bf57a1b2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.644442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282cab7-8340-4ec4-bb21-c9bf57a1b2a9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.657068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1024/2036] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:36.661939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.673564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891904-1ea9-4c5a-8c99-68e870028bf8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.678408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.679081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.679379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.679620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] PATH_INFO: `/v3/projects/a93ea819b3a04775a41b33b954fe0e8f/users/fbaab644a99b4546be231deac34dc754/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.695044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93ea819b3a04775a41b33b954fe0e8f, user_id=fbaab644a99b4546be231deac34dc754, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.697695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.713384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1013/2037] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:36 2022] PUT /identity/v3/projects/a93ea819b3a04775a41b33b954fe0e8f/users/fbaab644a99b4546be231deac34dc754/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:36.725427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.725708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.726017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.753071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] MFA Rules not processed for user `fbaab644a99b4546be231deac34dc754`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:36.754320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:36.824721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1025/2038] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:36.838229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:36.840246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:36.901995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19a5df-5ac3-420b-989a-ff83b1e7787a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:36.905592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:36.906230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:36.906508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:36.906779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:36.915081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:36.916438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:36.925226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1014/2039] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:46.821589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:46.921827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:52.022880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:52.024563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:52.104227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204ff2a-7f1a-42d5-a093-ad69bd270b21 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:52.108959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:52.109583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:52.109887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:52.110170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] PATH_INFO: `/v3/users/18297969a96a499da21f807217a3528b` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:52.111451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=18297969a96a499da21f807217a3528b)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:52.113391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:52.183002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18297969a96a499da21f807217a3528b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.192276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18297969a96a499da21f807217a3528b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.192897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18297969a96a499da21f807217a3528b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.204321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18297969a96a499da21f807217a3528b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.205521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:52.216808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8e625ede-f49b-4dc1-a8e5-438a4db2c70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18297969a96a499da21f807217a3528b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.217684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533a38c-45d5-44a2-8e08-612899f87e15 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:52.222590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:52.223232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:52.223512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:52.223786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] PATH_INFO: `/v3/users/fbaab644a99b4546be231deac34dc754` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:52.224922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbaab644a99b4546be231deac34dc754)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:52.226923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:52.230538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1026/2040] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:52 2022] DELETE /identity/v3/users/18297969a96a499da21f807217a3528b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:58:52.232270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:52.232558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:52.233727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:52.256194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:52.258566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:52.360118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1027/2041] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:52.371574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbaab644a99b4546be231deac34dc754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.373096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbaab644a99b4546be231deac34dc754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.378197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:52.383820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbaab644a99b4546be231deac34dc754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.389977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:52.395768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbaab644a99b4546be231deac34dc754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.422508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a05b47c8-c5c3-41cb-b53d-255264ac3924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbaab644a99b4546be231deac34dc754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:52.435166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1015/2042] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:58:52 2022] DELETE /identity/v3/users/fbaab644a99b4546be231deac34dc754 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:52.437104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:52.437482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:52.437890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:52.439619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:52.464120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:52.465693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:52.543438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb7459-0b15-44d4-8190-44141dbe135b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:52.599086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1016/2043] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:52.613281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:52.615002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:52.618287 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:52.619454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:52.619695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:52.619967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:52.622140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:52.631930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7fcaf-2084-4ece-b106-5d372892ae62 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:52.637201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:52.637645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:52.637964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:52.638358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:52.640501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:52.642035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:52.650149 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:52.652184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:52.652827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1028/2044] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:52.677329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1017/2045] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:58:53.150363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.151837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.163348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.165521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.183347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770e9c-8a45-4103-9c82-72dda1f029d0 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.183570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7af41-f299-4c4f-ae10-0f736fae7b6f service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.190924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.191437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.192116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.192360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.192631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] PATH_INFO: `/v3/projects/dd1f626d07ce4c84a8edbf5eb218e753` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.192872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.193382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.193893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd1f626d07ce4c84a8edbf5eb218e753)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.193988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] PATH_INFO: `/v3/projects/a93ea819b3a04775a41b33b954fe0e8f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.195385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93ea819b3a04775a41b33b954fe0e8f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.195656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.198464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.255045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd1f626d07ce4c84a8edbf5eb218e753', 'request_id': 'req-1f8216b5-d43c-4094-8652-a9da89967e2a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.261430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93ea819b3a04775a41b33b954fe0e8f', 'request_id': 'req-36ce2be4-ece3-4906-8430-4754f6729186'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.265246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd1f626d07ce4c84a8edbf5eb218e753', 'request_id': 'req-1f8216b5-d43c-4094-8652-a9da89967e2a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.270482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93ea819b3a04775a41b33b954fe0e8f', 'request_id': 'req-36ce2be4-ece3-4906-8430-4754f6729186'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.271932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:53.272328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd1f626d07ce4c84a8edbf5eb218e753', 'request_id': 'req-1f8216b5-d43c-4094-8652-a9da89967e2a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.279408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93ea819b3a04775a41b33b954fe0e8f', 'request_id': 'req-36ce2be4-ece3-4906-8430-4754f6729186'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.289502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] The token cache is being invalidate because project dd1f626d07ce4c84a8edbf5eb218e753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:53.289879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.290957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1f8216b5-d43c-4094-8652-a9da89967e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.292829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1029/2046] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:53 2022] DELETE /identity/v3/projects/dd1f626d07ce4c84a8edbf5eb218e753 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:53.295841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] The token cache is being invalidate because project a93ea819b3a04775a41b33b954fe0e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:58:53.297281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.298261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36ce2be4-ece3-4906-8430-4754f6729186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:58:53.300425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1018/2047] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:58:53 2022] DELETE /identity/v3/projects/a93ea819b3a04775a41b33b954fe0e8f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:53.301828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52c39-76b6-4466-ad26-4cab98e540cc None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.302126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52c39-76b6-4466-ad26-4cab98e540cc None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.302427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52c39-76b6-4466-ad26-4cab98e540cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.305605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746cd7-7d33-43c4-ac49-b5351c8d8a38 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.306114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746cd7-7d33-43c4-ac49-b5351c8d8a38 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.306447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746cd7-7d33-43c4-ac49-b5351c8d8a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.333555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-7aa52c39-76b6-4466-ad26-4cab98e540cc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:53.335174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7aa52c39-76b6-4466-ad26-4cab98e540cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.338693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e1746cd7-7d33-43c4-ac49-b5351c8d8a38 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:53.341512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e1746cd7-7d33-43c4-ac49-b5351c8d8a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.413454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1030/2048] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:53.421503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea73e-feb8-48f6-bd38-7816c00a7696 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.421817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea73e-feb8-48f6-bd38-7816c00a7696 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.422097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea73e-feb8-48f6-bd38-7816c00a7696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.423260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1019/2049] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:53.431065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d08db4-8ba0-4b6f-b02c-44422d2bdf57 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.431476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d08db4-8ba0-4b6f-b02c-44422d2bdf57 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.431818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d08db4-8ba0-4b6f-b02c-44422d2bdf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.436522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:53.457515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-603ea73e-feb8-48f6-bd38-7816c00a7696 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:53.459013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-603ea73e-feb8-48f6-bd38-7816c00a7696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.460044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-66d08db4-8ba0-4b6f-b02c-44422d2bdf57 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:53.461794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66d08db4-8ba0-4b6f-b02c-44422d2bdf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.470717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1020/2050] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:53.474252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1031/2051] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:53.474965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d08db4-8ba0-4b6f-b02c-44422d2bdf57 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.478143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ea73e-feb8-48f6-bd38-7816c00a7696 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.485414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d08db4-8ba0-4b6f-b02c-44422d2bdf57 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.488042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ea73e-feb8-48f6-bd38-7816c00a7696 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.490414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.491123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.491406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.491693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.492331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:53.492631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.493124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.493726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.494040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.494276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.494384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.494969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:58:53.496925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.497055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.509814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1021/2052] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:58:53.513811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.517823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1032/2053] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:58:53.521517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.525935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398cab-0b61-4ada-a994-45fb47b20a94 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.530794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.531426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.531754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.531959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.532630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.533137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a82c-8f0b-4d78-b449-4de6adb27528 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.534583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.540304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.541223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.541634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.542503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.543382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.545288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.571023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1022/2054] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:58:53.572072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.583264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1033/2055] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:58:53.587538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.660002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4196d-ecbc-4f08-b510-11cd71a42291 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.668248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.668248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.668529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.668625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.669210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.671778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.672036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b386218-6ab1-456f-9ce0-1fe43eca058d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.676896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.677499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.677809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.678064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.678688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.680565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.695303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1023/2056] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:58:53.699683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.707944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1034/2057] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:58:53.711762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.712824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e43840-c432-4f91-8bd6-b08c6ea73e1f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.719427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.720226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.720604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.720972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.721742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.723615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.727208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8da75-e623-4d91-afc8-20714471ee2b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.735759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.736379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.736774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1024/2058] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:58:53.737687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.738002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.738708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.740820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.740957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.751296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1a77-9370-4e5a-8729-d5d9b10994d7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.758215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1035/2059] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:53.759747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.761860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.762490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.762777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.763049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.763696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.766110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.772857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb04690-839a-4df8-944b-d0e554c328c6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.777638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.778275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.778421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1025/2060] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:53.778638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.778806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.779523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.781258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.782747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.792476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1036/2061] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:58:53.793048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cdd9c-7f1f-4744-a5da-1fe0b8fe1e21 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.797198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.798651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ff6e-b7d2-43a2-8559-8a4f356b3b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.799421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ff6e-b7d2-43a2-8559-8a4f356b3b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.799669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ff6e-b7d2-43a2-8559-8a4f356b3b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.800066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ff6e-b7d2-43a2-8559-8a4f356b3b3f admin admin] PATH_INFO: `/v3/projects/b0aec947283643dd9dfdce33a7b2799e/users/828f876065e4413f9447efb451cb5371/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.804025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:53.808910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb2d5-1ffc-4bcb-bfbc-3fd300c22a8d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:53.815128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e52c0-2b88-43f3-910a-2e9e3c052076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:53.815803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e52c0-2b88-43f3-910a-2e9e3c052076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.816131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e52c0-2b88-43f3-910a-2e9e3c052076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.816354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e52c0-2b88-43f3-910a-2e9e3c052076 admin admin] PATH_INFO: `/v3/projects/5db10dff8c774129a8d1797ce72fe258/users/970aa87c55284c32ba2c4511b0d2ba93/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.819962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ff6e-b7d2-43a2-8559-8a4f356b3b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0aec947283643dd9dfdce33a7b2799e, user_id=828f876065e4413f9447efb451cb5371, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.822716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ff6e-b7d2-43a2-8559-8a4f356b3b3f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.833331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e52c0-2b88-43f3-910a-2e9e3c052076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db10dff8c774129a8d1797ce72fe258, user_id=970aa87c55284c32ba2c4511b0d2ba93, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:53.836115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e52c0-2b88-43f3-910a-2e9e3c052076 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:53.836847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1026/2062] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:53 2022] PUT /identity/v3/projects/b0aec947283643dd9dfdce33a7b2799e/users/828f876065e4413f9447efb451cb5371/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:53.842252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.842556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.842830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.850334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1037/2063] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:58:53 2022] PUT /identity/v3/projects/5db10dff8c774129a8d1797ce72fe258/users/970aa87c55284c32ba2c4511b0d2ba93/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:58:53.861754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:53.862071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:53.862351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:53.862472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:53.863652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.891611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] MFA Rules not processed for user `970aa87c55284c32ba2c4511b0d2ba93`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:53.892831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.925733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1027/2064] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:53.937934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.939518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:53.959785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:53.972508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1038/2065] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:58:53.988315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:53.990886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:54.019933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6826e57-e1fc-4d28-b939-8fdfdf27dbe0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:54.025408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531cc82-dc52-4d24-8090-9c54e5133800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:54.026190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531cc82-dc52-4d24-8090-9c54e5133800 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:54.026596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531cc82-dc52-4d24-8090-9c54e5133800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:54.027000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531cc82-dc52-4d24-8090-9c54e5133800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:54.040141 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531cc82-dc52-4d24-8090-9c54e5133800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:54.042992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531cc82-dc52-4d24-8090-9c54e5133800 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:54.058359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1028/2066] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:54.073256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c46536-776e-4660-8b6d-422355c34a43 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:54.080116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:54.080778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:54.081050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:54.081320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:54.105493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:54.107396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:54.126006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1039/2067] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:54.847193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d97c5-de1b-456e-9eeb-2bf1272cd53c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:54.847779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d97c5-de1b-456e-9eeb-2bf1272cd53c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:54.847779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d97c5-de1b-456e-9eeb-2bf1272cd53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:54.886739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5e4d97c5-de1b-456e-9eeb-2bf1272cd53c None None] MFA Rules not processed for user `828f876065e4413f9447efb451cb5371`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:58:54.887990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e4d97c5-de1b-456e-9eeb-2bf1272cd53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:54.976470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1029/2068] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:58:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:58:54.989338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:54.991209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:55.006106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89232ea6-ab59-4733-9b47-243d883ce21d service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:55.012402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c90c8-dd82-454d-b9de-32d2ab7bc28e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:55.012926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c90c8-dd82-454d-b9de-32d2ab7bc28e service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:55.013211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c90c8-dd82-454d-b9de-32d2ab7bc28e service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:55.014742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c90c8-dd82-454d-b9de-32d2ab7bc28e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:55.026113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c90c8-dd82-454d-b9de-32d2ab7bc28e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:55.027617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c90c8-dd82-454d-b9de-32d2ab7bc28e service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:55.048257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1040/2069] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:00.680435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee1199-eb9d-4516-8c72-80369afe7d88 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:00.681146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee1199-eb9d-4516-8c72-80369afe7d88 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:00.681146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee1199-eb9d-4516-8c72-80369afe7d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:00.706432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-66ee1199-eb9d-4516-8c72-80369afe7d88 None None] MFA Rules not processed for user `828f876065e4413f9447efb451cb5371`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:00.706794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66ee1199-eb9d-4516-8c72-80369afe7d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:00.719015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1030/2070] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:04.916320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cfc33-fef7-46cf-84cf-c4de497e0c7c None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:04.916922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cfc33-fef7-46cf-84cf-c4de497e0c7c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:04.916922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cfc33-fef7-46cf-84cf-c4de497e0c7c None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:04.918435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1041/2071] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:59:04.924977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:04.925591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:04.925591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:04.950903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:04.952814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:04.960969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:04.991358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cfc33-fef7-46cf-84cf-c4de497e0c7c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:04.994586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7f8cfc33-fef7-46cf-84cf-c4de497e0c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:05.021984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cfc33-fef7-46cf-84cf-c4de497e0c7c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:05.027219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:05.027848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:05.028127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:05.028508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] PATH_INFO: `/v3/users/1015b24d41f446069b0e0884b92a1772` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:05.029627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1015b24d41f446069b0e0884b92a1772)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:05.031576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:05.050018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1031/2072] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:59:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:05.060443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:05.062768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:05.082572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9b975-c1fb-42bc-9bd8-fcbc74910158 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:05.089607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:05.090266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:05.090735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:05.091013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:05.100646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:05.102273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:05.105622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1015b24d41f446069b0e0884b92a1772'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.114359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1032/2073] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:05.115222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1015b24d41f446069b0e0884b92a1772'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.115966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1015b24d41f446069b0e0884b92a1772'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.124325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1015b24d41f446069b0e0884b92a1772'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.141259 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db27cea2-d074-4bd4-b89b-bd629ab5325c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1015b24d41f446069b0e0884b92a1772'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.160556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1042/2074] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:04 2022] DELETE /identity/v3/users/1015b24d41f446069b0e0884b92a1772 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:05.410168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:05.435375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3c31d-bebe-4876-9bfe-b29d96283dbe service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:05.527598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:05.528946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:05.528946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:05.528946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] PATH_INFO: `/v3/projects/ced3cefcaf434541a38dd01270076279` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:05.530841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced3cefcaf434541a38dd01270076279)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:05.532551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:05.598870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced3cefcaf434541a38dd01270076279', 'request_id': 'req-67cb41d6-b47e-4c1f-a363-6b3123690c78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.609700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced3cefcaf434541a38dd01270076279', 'request_id': 'req-67cb41d6-b47e-4c1f-a363-6b3123690c78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.631207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced3cefcaf434541a38dd01270076279', 'request_id': 'req-67cb41d6-b47e-4c1f-a363-6b3123690c78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.652994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] The token cache is being invalidate because project ced3cefcaf434541a38dd01270076279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:05.653419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.654350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-67cb41d6-b47e-4c1f-a363-6b3123690c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:05.657835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1033/2075] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:05 2022] DELETE /identity/v3/projects/ced3cefcaf434541a38dd01270076279 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:05.662610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39413-bb39-4343-b2b4-2a988b733ae9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:05.663209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39413-bb39-4343-b2b4-2a988b733ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:05.663703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39413-bb39-4343-b2b4-2a988b733ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:05.690124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-55e39413-bb39-4343-b2b4-2a988b733ae9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:05.691357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-55e39413-bb39-4343-b2b4-2a988b733ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:05.752012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:05.775394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1043/2076] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:05.777182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16774ce-561b-4293-9906-de47562f987c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:05.777497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16774ce-561b-4293-9906-de47562f987c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:05.777748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16774ce-561b-4293-9906-de47562f987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:05.814283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c16774ce-561b-4293-9906-de47562f987c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:05.815574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c16774ce-561b-4293-9906-de47562f987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:05.829555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1034/2077] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:05.833444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e39413-bb39-4343-b2b4-2a988b733ae9 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:05.847321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e39413-bb39-4343-b2b4-2a988b733ae9 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:05.853121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:05.854432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:05.854790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:05.855136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:05.856480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:05.856796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:05.858955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:05.876673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1044/2078] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:59:05.878305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16774ce-561b-4293-9906-de47562f987c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:05.888997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16774ce-561b-4293-9906-de47562f987c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:05.902191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:05.902978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:05.903297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:05.903628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:05.904267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:05.907499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:05.926174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:05.943311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1035/2079] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:05 2022] POST /identity/v3/projects => generated 403 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:05.947298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:05.958936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8659e3c-dc04-4bda-9a67-aef28cc7eaf9 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:06.045653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:06.046331 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:06.046789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:06.047199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:06.047836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:06.050086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:06.083116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1045/2080] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:05 2022] POST /identity/v3/users => generated 302 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:06.087403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:06.099269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3274d93-8f48-4122-8bea-d795f5545b90 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:06.111775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:06.112413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:06.113029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:06.113486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:06.114429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:06.116430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:06.134999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1036/2081] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:06.138906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:06.151897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e0bd1-e07a-43c3-a2c8-771793a736b8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:06.160651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbc1ea-831e-4815-99e8-82900394a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:06.161654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc1ea-831e-4815-99e8-82900394a5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:06.161962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc1ea-831e-4815-99e8-82900394a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:06.162249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc1ea-831e-4815-99e8-82900394a5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:06.162874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbc1ea-831e-4815-99e8-82900394a5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:06.164624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbc1ea-831e-4815-99e8-82900394a5eb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:06.176870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1046/2082] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:06.181101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:06.192732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c1cdf-46c8-45bd-9dc4-cb4334b6c8f5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:06.197474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:06.198239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:06.198548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:06.198738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] PATH_INFO: `/v3/projects/e3e3bb86a20e4e72b164a98b67774f27/users/2694b25b7a1848daa5c3c5c967ed3139/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:06.215709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e3bb86a20e4e72b164a98b67774f27, user_id=2694b25b7a1848daa5c3c5c967ed3139, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:06.219307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:06.232283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1037/2083] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:06 2022] PUT /identity/v3/projects/e3e3bb86a20e4e72b164a98b67774f27/users/2694b25b7a1848daa5c3c5c967ed3139/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:06.238061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc41a9c-7dcb-4b66-a202-48f90682096f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:06.238343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc41a9c-7dcb-4b66-a202-48f90682096f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:06.238616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc41a9c-7dcb-4b66-a202-48f90682096f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:06.258929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4cc41a9c-7dcb-4b66-a202-48f90682096f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:06.260098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4cc41a9c-7dcb-4b66-a202-48f90682096f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:06.337397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1047/2084] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:06.376445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:06.378335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:06.470643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f6c1a-ec30-43dd-be87-16bfefe7beda admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:06.478423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:06.479282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:06.479553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:06.480556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:06.503527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:06.505249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:06.519516 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1038/2085] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:07.375022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df743-a124-4221-a7b2-0d67a00d843d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:07.375962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df743-a124-4221-a7b2-0d67a00d843d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:07.376389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df743-a124-4221-a7b2-0d67a00d843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:07.411715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-014df743-a124-4221-a7b2-0d67a00d843d None None] MFA Rules not processed for user `2694b25b7a1848daa5c3c5c967ed3139`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:07.413423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-014df743-a124-4221-a7b2-0d67a00d843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:07.501456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:07.538186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1048/2086] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:07 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:07.543027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:07.557383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e367a-3dee-4ea6-9b92-f175462af7d0 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:07.564062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:07.564950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:07.565379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:07.565909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:07.566692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:07.569261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:07.615091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1039/2087] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:07 2022] POST /identity/v3/projects => generated 405 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:07.620240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014df743-a124-4221-a7b2-0d67a00d843d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:07.637834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014df743-a124-4221-a7b2-0d67a00d843d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:07.734759 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:07.735445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:07.735730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:07.736005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:07.736879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:07.739632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:07.782480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1049/2088] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:07 2022] POST /identity/v3/users => generated 309 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:07.790757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:07.807957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b300118-43d8-41ca-a70a-0101b004776b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:07.819900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:07.820781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:07.821211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:07.821635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:07.822697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:07.825035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:07.839928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1040/2089] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:07.846641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:07.860698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633f9c2-fa7e-4512-9cf5-b7c7e1437936 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:07.881726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:07.882641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:07.882983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:07.883325 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] PATH_INFO: `/v3/projects/74b35b5d81104abfa4b79e5d5dfce3fe/users/abc359aedcda4de0905b946a2c2d8cc6/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:07.905062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b35b5d81104abfa4b79e5d5dfce3fe, user_id=abc359aedcda4de0905b946a2c2d8cc6, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:07.908001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:07.924529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1050/2090] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:07 2022] PUT /identity/v3/projects/74b35b5d81104abfa4b79e5d5dfce3fe/users/abc359aedcda4de0905b946a2c2d8cc6/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:07.928570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:07.930296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:07.932286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:07.943220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d689b8-e070-43cd-8105-43502abf6ef2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:07.998297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5d5e8-b418-4666-a63c-17917c4af389 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:08.007413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:08.008291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.008660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.008991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.009702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:08.011525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:08.025233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1051/2091] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:08.030177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:08.030321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:08.030896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.031148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.031432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] PATH_INFO: `/v3/users/892ba3dad26b4ce6ae17ec425acc5087` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.032777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=892ba3dad26b4ce6ae17ec425acc5087)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:08.035072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:08.042673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c671338-dbc8-470b-889f-444994d1dfb6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:08.049942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ba97b-96fa-4a32-8583-f278d999d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:08.050616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ba97b-96fa-4a32-8583-f278d999d5da admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.050898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ba97b-96fa-4a32-8583-f278d999d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.051174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ba97b-96fa-4a32-8583-f278d999d5da admin admin] PATH_INFO: `/v3/projects/74b35b5d81104abfa4b79e5d5dfce3fe/users/abc359aedcda4de0905b946a2c2d8cc6/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.053144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ba97b-96fa-4a32-8583-f278d999d5da admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b35b5d81104abfa4b79e5d5dfce3fe, user_id=abc359aedcda4de0905b946a2c2d8cc6, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:08.055812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ba97b-96fa-4a32-8583-f278d999d5da admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:08.071239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1052/2092] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:08 2022] PUT /identity/v3/projects/74b35b5d81104abfa4b79e5d5dfce3fe/users/abc359aedcda4de0905b946a2c2d8cc6/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:08.077732 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:08.095146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892ba3dad26b4ce6ae17ec425acc5087'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.096938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892ba3dad26b4ce6ae17ec425acc5087'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.108698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892ba3dad26b4ce6ae17ec425acc5087'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.118699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892ba3dad26b4ce6ae17ec425acc5087'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.137510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892ba3dad26b4ce6ae17ec425acc5087'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.147213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1041/2093] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:07 2022] DELETE /identity/v3/users/892ba3dad26b4ce6ae17ec425acc5087 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:08.151780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.152286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.152741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.174522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:08.176174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:08.217563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:08.251216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1053/2094] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:08.258017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:08.259343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:08.332679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7220983-41f3-4027-8e49-bbb70b7654fe admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:08.342662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa9198-9389-4eaa-8e76-1370084d6d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:08.343305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9198-9389-4eaa-8e76-1370084d6d18 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.343701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9198-9389-4eaa-8e76-1370084d6d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.343980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9198-9389-4eaa-8e76-1370084d6d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.357023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa9198-9389-4eaa-8e76-1370084d6d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:08.359154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa9198-9389-4eaa-8e76-1370084d6d18 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:08.380309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1042/2095] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:08.759659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:08.762723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:08.785518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad78c0-798b-4c38-86d4-8380ef8f7329 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:08.805311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:08.807187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.808196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.808553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] PATH_INFO: `/v3/projects/cacc64a1dc88491c90a4e3552c85a482` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.810745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cacc64a1dc88491c90a4e3552c85a482)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:08.814022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:08.900852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cacc64a1dc88491c90a4e3552c85a482', 'request_id': 'req-65e0cd04-5a12-45f8-8219-b881ff0cad65'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.913250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305f89c-a61a-41de-96cc-f33e0a0a79e7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.913695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305f89c-a61a-41de-96cc-f33e0a0a79e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.913889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305f89c-a61a-41de-96cc-f33e0a0a79e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.918018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cacc64a1dc88491c90a4e3552c85a482', 'request_id': 'req-65e0cd04-5a12-45f8-8219-b881ff0cad65'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.928036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cacc64a1dc88491c90a4e3552c85a482', 'request_id': 'req-65e0cd04-5a12-45f8-8219-b881ff0cad65'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.950373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] The token cache is being invalidate because project cacc64a1dc88491c90a4e3552c85a482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:08.950812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.954171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-65e0cd04-5a12-45f8-8219-b881ff0cad65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:08.960537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1054/2096] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:08 2022] DELETE /identity/v3/projects/cacc64a1dc88491c90a4e3552c85a482 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:08.962815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcb767-e4a9-4d35-9f0a-3761bf757122 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:08.963272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcb767-e4a9-4d35-9f0a-3761bf757122 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:08.963653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcb767-e4a9-4d35-9f0a-3761bf757122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:08.998582 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-67dcb767-e4a9-4d35-9f0a-3761bf757122 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:09.000683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-67dcb767-e4a9-4d35-9f0a-3761bf757122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.004346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e305f89c-a61a-41de-96cc-f33e0a0a79e7 None None] MFA Rules not processed for user `abc359aedcda4de0905b946a2c2d8cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:09.005614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e305f89c-a61a-41de-96cc-f33e0a0a79e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.150655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1043/2097] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:09.170858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1055/2098] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:09.189864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b45df4-91fb-4318-b114-4f7b5e65d766 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.189864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b45df4-91fb-4318-b114-4f7b5e65d766 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.189864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b45df4-91fb-4318-b114-4f7b5e65d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.213598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-53b45df4-91fb-4318-b114-4f7b5e65d766 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:09.215260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-53b45df4-91fb-4318-b114-4f7b5e65d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.225390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1044/2099] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:09.229305 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcb767-e4a9-4d35-9f0a-3761bf757122 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.242068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcb767-e4a9-4d35-9f0a-3761bf757122 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.247455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.248936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.249330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.249670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.250461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:09.250750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.252845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.281456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1056/2100] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:59:09.285551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b45df4-91fb-4318-b114-4f7b5e65d766 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.295559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b45df4-91fb-4318-b114-4f7b5e65d766 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.308049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.308049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.308049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.308049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.308424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.308424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.340409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1045/2101] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:09 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:09.344659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.357051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb2076-c0a9-44fc-b594-020c2c868f8c admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.452693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:09.475934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.476556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.476846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.477121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.477794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.479842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.530704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1057/2102] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:09 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:09.532692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.536589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.546629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.562326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e33a30-096d-4f56-8816-56c499b86274 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.569356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.570340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.570959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.571383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.571709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b8bba-3c7c-44d9-a096-25432eec144b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.572572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.574765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.577409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:09.577978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.578976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.579483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.579906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] PATH_INFO: `/v3/users/a4fb223600154d618d2313a2d88a40aa` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.581842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4fb223600154d618d2313a2d88a40aa)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.584236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.606862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1046/2103] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:09.608450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.632496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca3fc8-5ec8-4195-9f81-9a307486db2c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.643986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.646735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.647023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.647360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.648015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.651920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.668557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4fb223600154d618d2313a2d88a40aa'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:09.677057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1047/2104] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:09.681802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.681967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4fb223600154d618d2313a2d88a40aa'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:09.682900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4fb223600154d618d2313a2d88a40aa'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:09.696011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4fb223600154d618d2313a2d88a40aa'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:09.698973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a9c08-b2a0-407b-8d15-bfab403b0045 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.704710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decbf1a-e9f4-498f-97be-373f3f74d77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.705802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decbf1a-e9f4-498f-97be-373f3f74d77e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.705989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decbf1a-e9f4-498f-97be-373f3f74d77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.706238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decbf1a-e9f4-498f-97be-373f3f74d77e admin admin] PATH_INFO: `/v3/projects/08ad952b74d044dd879dbb28ab2231a1/users/4715f245c74c4a58ac5dc27d825d6260/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.725058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-29f4aea0-439e-449c-87f9-cf6a51739811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4fb223600154d618d2313a2d88a40aa'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:09.729641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decbf1a-e9f4-498f-97be-373f3f74d77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ad952b74d044dd879dbb28ab2231a1, user_id=4715f245c74c4a58ac5dc27d825d6260, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.738986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decbf1a-e9f4-498f-97be-373f3f74d77e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.740270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1058/2105] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:09 2022] DELETE /identity/v3/users/a4fb223600154d618d2313a2d88a40aa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:09.744554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.745619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.745967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.763002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1048/2106] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:09 2022] PUT /identity/v3/projects/08ad952b74d044dd879dbb28ab2231a1/users/4715f245c74c4a58ac5dc27d825d6260/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:09.774857 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.775900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.776276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.787141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:09.788405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.809377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] MFA Rules not processed for user `4715f245c74c4a58ac5dc27d825d6260`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:09.811534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.911680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1049/2107] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:09.917299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1059/2108] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:09.917581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.922627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.928101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:09.929599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:09.939896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa134-8ee7-4b55-8f7b-d44bdad2e4f5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:09.945258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:09.946177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:09.946660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:09.947114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:09.947929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:09.950104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:09.988472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1050/2109] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:09 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:09.994667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.041598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4d3a0-5810-425e-a606-6f7c38f57875 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.051762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:10.119509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.120434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.120875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.121294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.128183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c81ee-f9b1-4f49-aae2-aabbca4fc7d6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.155158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13beb-0794-426b-8631-0388626e30ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.155158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13beb-0794-426b-8631-0388626e30ec admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.155158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13beb-0794-426b-8631-0388626e30ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.155158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13beb-0794-426b-8631-0388626e30ec admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.155483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf13beb-0794-426b-8631-0388626e30ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.155483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf13beb-0794-426b-8631-0388626e30ec admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:10.155618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.156128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:10.191239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1051/2111] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:09 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:10.192646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dfc90-9593-44df-a6e6-1fc6a5af2425 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.192940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dfc90-9593-44df-a6e6-1fc6a5af2425 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.193279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dfc90-9593-44df-a6e6-1fc6a5af2425 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.197322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1052/2112] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 11:59:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:59:10.197711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1060/2112] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:10.202852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dfc90-9593-44df-a6e6-1fc6a5af2425 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.231121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dfc90-9593-44df-a6e6-1fc6a5af2425 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.242904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20800ba1-a8ca-4966-9b0e-3035f3d1b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.243530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800ba1-a8ca-4966-9b0e-3035f3d1b012 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.245444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800ba1-a8ca-4966-9b0e-3035f3d1b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.247427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800ba1-a8ca-4966-9b0e-3035f3d1b012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.249257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20800ba1-a8ca-4966-9b0e-3035f3d1b012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.261909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20800ba1-a8ca-4966-9b0e-3035f3d1b012 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:10.280545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1053/2113] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:10.285200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.289176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.289484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.289777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.321854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ce24c-9d54-4125-b4db-e5a67e612647 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.330764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:10.332573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:10.337341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.337980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:10.338219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.338497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.338713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.341019 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.342210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:10.367892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1061/2114] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:10.370453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.392271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d87c-b0af-45aa-ae1a-df101bcca7f1 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.399274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8f8df-9797-4e8a-bc5e-60bb768a54e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.399991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8f8df-9797-4e8a-bc5e-60bb768a54e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.400271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8f8df-9797-4e8a-bc5e-60bb768a54e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.400873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8f8df-9797-4e8a-bc5e-60bb768a54e5 admin admin] PATH_INFO: `/v3/projects/552178b304a64ee0b1b2890bf58d866d/users/5dc5a24bc6c24e75a34871837378b6ca/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.431375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8f8df-9797-4e8a-bc5e-60bb768a54e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552178b304a64ee0b1b2890bf58d866d, user_id=5dc5a24bc6c24e75a34871837378b6ca, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.452415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8f8df-9797-4e8a-bc5e-60bb768a54e5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:10.483523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1062/2115] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:10 2022] PUT /identity/v3/projects/552178b304a64ee0b1b2890bf58d866d/users/5dc5a24bc6c24e75a34871837378b6ca/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:10.485779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128f72-844e-4f3b-809d-49fa453852d9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.486215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128f72-844e-4f3b-809d-49fa453852d9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.486344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128f72-844e-4f3b-809d-49fa453852d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.517269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1054/2116] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:10.555035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.557228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:10.564180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-89128f72-844e-4f3b-809d-49fa453852d9 None None] MFA Rules not processed for user `5dc5a24bc6c24e75a34871837378b6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:10.574270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-89128f72-844e-4f3b-809d-49fa453852d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:10.601718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ea889-0bf8-4c5a-b9cc-da1dbcbf78fa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.760695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.762423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.762871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.763456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.766931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1063/2117] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:10.771651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89128f72-844e-4f3b-809d-49fa453852d9 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.790238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.793805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:10.823881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1055/2118] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:10.895740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89128f72-844e-4f3b-809d-49fa453852d9 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.905504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.906558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.907017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.907488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.908894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.912477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:10.935108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.936687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:10.944682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:10.965553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469df3a-e112-4d84-b401-aedc9e28523c service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:10.973869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1064/2119] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:10 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:10.976064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:10.978686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:10.980454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:10.980767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:10.981046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] PATH_INFO: `/v3/projects/3379b4d9719d4c66a277818f728942fe` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:10.982977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3379b4d9719d4c66a277818f728942fe)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:10.985274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.054084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbcd55-e459-4913-81e8-3344bf8f3688 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.054336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3379b4d9719d4c66a277818f728942fe', 'request_id': 'req-38a6fd69-4c72-4863-9bb5-9841b6500311'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.061707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3379b4d9719d4c66a277818f728942fe', 'request_id': 'req-38a6fd69-4c72-4863-9bb5-9841b6500311'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.069868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3379b4d9719d4c66a277818f728942fe', 'request_id': 'req-38a6fd69-4c72-4863-9bb5-9841b6500311'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.085756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] The token cache is being invalidate because project 3379b4d9719d4c66a277818f728942fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:11.086642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.087052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.088706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1056/2120] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:10 2022] DELETE /identity/v3/projects/3379b4d9719d4c66a277818f728942fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:11.093054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.094624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.121344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:11.131259 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.132501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.132903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.133303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.134064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.136126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.168322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1065/2121] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:10 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:11.169411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.172283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.184842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80049b90-bc48-46a1-97d8-adb48d352523 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.188268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6fd69-4c72-4863-9bb5-9841b6500311 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.189710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.190359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.190637 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.190908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.191707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.193298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.195343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.196400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.196631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.196902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] PATH_INFO: `/v3/users/8d7947b8ded748e59b5d98c35a5f1c99` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.198175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7947b8ded748e59b5d98c35a5f1c99)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.200579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.205091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1066/2122] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:11.209529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.221228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d332818-d5c0-46c5-b0ce-8ffc66218f57 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.226037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.226700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.226979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.227252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] PATH_INFO: `/v3/projects/cd7ccbd1083846ccb0db0e32954e925f/users/27386cf46d364224855efa2a3e889c9a/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.249085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7ccbd1083846ccb0db0e32954e925f, user_id=27386cf46d364224855efa2a3e889c9a, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.252302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.260464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7947b8ded748e59b5d98c35a5f1c99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.261103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7947b8ded748e59b5d98c35a5f1c99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.266823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1067/2123] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:11 2022] PUT /identity/v3/projects/cd7ccbd1083846ccb0db0e32954e925f/users/27386cf46d364224855efa2a3e889c9a/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:11.267629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.269208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.271446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7947b8ded748e59b5d98c35a5f1c99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.283895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7947b8ded748e59b5d98c35a5f1c99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.300903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7947b8ded748e59b5d98c35a5f1c99'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.307921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1057/2124] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:11 2022] DELETE /identity/v3/users/8d7947b8ded748e59b5d98c35a5f1c99 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:11.309451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.313977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.333696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:11.348730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84287227-880d-4017-88ce-4e5d2c19a101 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.397258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d32786-7715-4ec3-bc0e-a306edbbb2e9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.404839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.405684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.406111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.406447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.407157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.409190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.427203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1058/2125] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:11.427365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.427365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.427365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.427365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] PATH_INFO: `/v3/users/2694b25b7a1848daa5c3c5c967ed3139` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.428160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2694b25b7a1848daa5c3c5c967ed3139)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.428655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.430745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.439835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78903138-78d2-4016-9432-9bb87c667021 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.444829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989c1d6-13a8-4069-bf26-0a7bbe09bbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.445508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989c1d6-13a8-4069-bf26-0a7bbe09bbf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.445821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989c1d6-13a8-4069-bf26-0a7bbe09bbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.446102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989c1d6-13a8-4069-bf26-0a7bbe09bbf7 admin admin] PATH_INFO: `/v3/projects/cd7ccbd1083846ccb0db0e32954e925f/users/27386cf46d364224855efa2a3e889c9a/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.448284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989c1d6-13a8-4069-bf26-0a7bbe09bbf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7ccbd1083846ccb0db0e32954e925f, user_id=27386cf46d364224855efa2a3e889c9a, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.451349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989c1d6-13a8-4069-bf26-0a7bbe09bbf7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.472785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1059/2126] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:11 2022] PUT /identity/v3/projects/cd7ccbd1083846ccb0db0e32954e925f/users/27386cf46d364224855efa2a3e889c9a/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:11.474802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.475220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.475965 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.524814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2694b25b7a1848daa5c3c5c967ed3139'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.537811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2694b25b7a1848daa5c3c5c967ed3139'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.539605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2694b25b7a1848daa5c3c5c967ed3139'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.540265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] MFA Rules not processed for user `27386cf46d364224855efa2a3e889c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:11.543016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.550191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2694b25b7a1848daa5c3c5c967ed3139'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.560881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:11.582670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2694b25b7a1848daa5c3c5c967ed3139'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:11.591352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1068/2127] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:11 2022] DELETE /identity/v3/users/2694b25b7a1848daa5c3c5c967ed3139 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:11.713095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1060/2128] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:11.726289 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.727748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.734436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:11.738105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.823560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708c913-fb53-4985-8e49-5dc873199b57 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.828633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.829334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.829671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.830337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.831980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:11.883694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7797d-5387-47fa-9776-81f74017f98b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:11.904854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:11.962318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.963935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:11.985645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:11.986368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:11.986717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:11.986920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] PATH_INFO: `/v3/projects/2d959f285cff48d0968a6a3f709f6f98` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:11.989977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d959f285cff48d0968a6a3f709f6f98)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:11.992561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.029956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1069/2129] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:12.030955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.032370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.047174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d09c1-8fb6-4b9e-9fb8-f0f41ba86f7a service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.052207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.052924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.053269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.053610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] PATH_INFO: `/v3/projects/e3e3bb86a20e4e72b164a98b67774f27` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.054906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e3bb86a20e4e72b164a98b67774f27)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.056942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.090483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d959f285cff48d0968a6a3f709f6f98', 'request_id': 'req-69496d76-9847-4e46-8c7d-814fd7181a57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.106042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d959f285cff48d0968a6a3f709f6f98', 'request_id': 'req-69496d76-9847-4e46-8c7d-814fd7181a57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.109519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:12.119865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d959f285cff48d0968a6a3f709f6f98', 'request_id': 'req-69496d76-9847-4e46-8c7d-814fd7181a57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.136330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e3bb86a20e4e72b164a98b67774f27', 'request_id': 'req-176b8ceb-5df0-42bf-be15-1986165d78e6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.139382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] The token cache is being invalidate because project 2d959f285cff48d0968a6a3f709f6f98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:12.139667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.140985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69496d76-9847-4e46-8c7d-814fd7181a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.149478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e3bb86a20e4e72b164a98b67774f27', 'request_id': 'req-176b8ceb-5df0-42bf-be15-1986165d78e6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.151779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1061/2130] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:11 2022] DELETE /identity/v3/projects/2d959f285cff48d0968a6a3f709f6f98 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:12.153435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd034-167b-492c-a099-a0fcb6f77b12 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.153826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd034-167b-492c-a099-a0fcb6f77b12 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.154702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd034-167b-492c-a099-a0fcb6f77b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.160801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e3bb86a20e4e72b164a98b67774f27', 'request_id': 'req-176b8ceb-5df0-42bf-be15-1986165d78e6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.179145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] The token cache is being invalidate because project e3e3bb86a20e4e72b164a98b67774f27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:12.179894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.180782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.182756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1070/2131] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:12 2022] DELETE /identity/v3/projects/e3e3bb86a20e4e72b164a98b67774f27 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:12.184807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-05bfd034-167b-492c-a099-a0fcb6f77b12 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:12.186675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-05bfd034-167b-492c-a099-a0fcb6f77b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.187424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.189342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.311767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1062/2132] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:12.315743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8ceb-5df0-42bf-be15-1986165d78e6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.322178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599bad-b30c-49de-949f-881822f58d73 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.322469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599bad-b30c-49de-949f-881822f58d73 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.322758 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599bad-b30c-49de-949f-881822f58d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.323311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.324087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.324368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.324696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] PATH_INFO: `/v3/users/abc359aedcda4de0905b946a2c2d8cc6` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.325869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc359aedcda4de0905b946a2c2d8cc6)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.327846 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.345730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-40599bad-b30c-49de-949f-881822f58d73 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:12.346609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40599bad-b30c-49de-949f-881822f58d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.361837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1063/2133] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:12.365549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40599bad-b30c-49de-949f-881822f58d73 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.377143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40599bad-b30c-49de-949f-881822f58d73 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.385228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.385874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.386152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.386429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.387079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:12.387389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.390090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.391756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc359aedcda4de0905b946a2c2d8cc6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.401058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc359aedcda4de0905b946a2c2d8cc6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.402093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc359aedcda4de0905b946a2c2d8cc6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.404940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1064/2134] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:59:12.407448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:12.409913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.411848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc359aedcda4de0905b946a2c2d8cc6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.412172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.429371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3ff6-391a-4e24-8cfa-c9b29c3566e5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.436935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.437551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.437849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.438134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.438761 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.440643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.442915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc359aedcda4de0905b946a2c2d8cc6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.450217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1071/2135] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:12 2022] DELETE /identity/v3/users/abc359aedcda4de0905b946a2c2d8cc6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:12.482928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1065/2136] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:12.486979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.572901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27ff4d-982a-4b00-9bbd-db97b7c5b3ef admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.634277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.634890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.635171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.635443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.636081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.638341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.650467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.652413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.671302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1072/2137] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:12.671499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf101c-9af2-476b-99be-86e3c2bf093b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.672417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.677487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.677795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.678081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.678363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] PATH_INFO: `/v3/projects/74b35b5d81104abfa4b79e5d5dfce3fe` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.679619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b35b5d81104abfa4b79e5d5dfce3fe)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.681550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.683720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc3911-0822-499c-bb12-d56a6bca981f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.689554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.690210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.690481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.690757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.691440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.693112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.704212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1073/2138] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:12.709873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.721107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33dfea-d501-45d4-b76c-fa5724d87a14 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.728176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.729545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.729887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.730147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.731088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.734026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.740318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b35b5d81104abfa4b79e5d5dfce3fe', 'request_id': 'req-15f9f6e7-7bac-4f78-b49e-1777417dbe22'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.746343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1074/2139] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:12.747856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b35b5d81104abfa4b79e5d5dfce3fe', 'request_id': 'req-15f9f6e7-7bac-4f78-b49e-1777417dbe22'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.751258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.755827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b35b5d81104abfa4b79e5d5dfce3fe', 'request_id': 'req-15f9f6e7-7bac-4f78-b49e-1777417dbe22'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.761400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede5843-fd04-472e-bc6e-e3a5154db15f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.771044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] The token cache is being invalidate because project 74b35b5d81104abfa4b79e5d5dfce3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:12.771405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.772128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15f9f6e7-7bac-4f78-b49e-1777417dbe22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:12.777199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1066/2140] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:12 2022] DELETE /identity/v3/projects/74b35b5d81104abfa4b79e5d5dfce3fe => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:12.779586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837f68-1aca-49d9-b2f7-b0fc7839a61f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.780006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837f68-1aca-49d9-b2f7-b0fc7839a61f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.780442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837f68-1aca-49d9-b2f7-b0fc7839a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.813268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-46837f68-1aca-49d9-b2f7-b0fc7839a61f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:12.815473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-46837f68-1aca-49d9-b2f7-b0fc7839a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.821843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:12.839817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad42ea2-9a57-48bd-bde2-34cca5f6988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:12.840668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad42ea2-9a57-48bd-bde2-34cca5f6988a admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.841068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad42ea2-9a57-48bd-bde2-34cca5f6988a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.841337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad42ea2-9a57-48bd-bde2-34cca5f6988a admin admin] PATH_INFO: `/v3/projects/55dd0abb05404c6aad89a27528d36b00/users/da31064c39b346f9b16d6f9b6fcc3e46/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.841724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:12.859678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad42ea2-9a57-48bd-bde2-34cca5f6988a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55dd0abb05404c6aad89a27528d36b00, user_id=da31064c39b346f9b16d6f9b6fcc3e46, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:12.863183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad42ea2-9a57-48bd-bde2-34cca5f6988a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:12.878705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1075/2141] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:12 2022] PUT /identity/v3/projects/55dd0abb05404c6aad89a27528d36b00/users/da31064c39b346f9b16d6f9b6fcc3e46/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:12.889644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.890031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.890807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.892671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1067/2142] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:12.900413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa828d-cba2-46b7-b8e3-5766b9b289be None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:12.900701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa828d-cba2-46b7-b8e3-5766b9b289be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:12.900977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa828d-cba2-46b7-b8e3-5766b9b289be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:12.921440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b0fa828d-cba2-46b7-b8e3-5766b9b289be None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:12.922678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b0fa828d-cba2-46b7-b8e3-5766b9b289be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.924921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] MFA Rules not processed for user `da31064c39b346f9b16d6f9b6fcc3e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:12.926207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:12.983886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1068/2143] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:12.987985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa828d-cba2-46b7-b8e3-5766b9b289be None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.992894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1076/2144] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:12.998324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:12.999332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa828d-cba2-46b7-b8e3-5766b9b289be None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:12.999878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:13.004128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.004738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.005018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.005292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.005978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:13.006287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.008184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.014203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b69f80-434b-418a-8eb1-dedef61e0aae None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.019594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.020342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.020463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.020785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.021298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1069/2145] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:59:13.024798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.025391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.026852 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.036675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e4a2a-a496-4573-9b6d-6ff49f630789 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.042304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.043127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.043408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.043680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.044303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.046266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.062132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1077/2146] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:12 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:13.063945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.080216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1070/2147] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:13.084094 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.131932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583ffd9-2841-4315-8eaf-edcc3c10abd5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.164344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8906d-9191-41bb-9ec8-9e4885a23861 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.168971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.169575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.170386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.170659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.171280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.173186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.186220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:13.196195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1071/2148] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/users => generated 278 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:13.200262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.200738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.201254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.201527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.201821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.202451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.204334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.210407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f022a-5344-4a7e-b5b7-519e925694cf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.215530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.216258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.216713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.217145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.218006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.220088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.226353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1078/2149] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:13.230316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.230459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1072/2150] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:13.234787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.240616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe179fe4-9630-42ab-9b8d-30d23424a778 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.244293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca71fc-3182-4ec9-9e74-c0551cad2c08 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.245158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.245749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.246053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.246321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.246964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.249186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.249920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.250511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.250794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.251073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.251718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.253291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.255848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:13.262581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1079/2151] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:13.264662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.267270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1073/2152] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:13.271124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.275087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd27d90-3fa4-4f48-a752-e9b45f9f7de9 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.280701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.281319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.281687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.282029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.282417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d011b7-7415-4ade-8bb7-423bc5813d1f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.282702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.284307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.286994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655552de-45c3-45a7-8047-814f7a9aa4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.287631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655552de-45c3-45a7-8047-814f7a9aa4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.287915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655552de-45c3-45a7-8047-814f7a9aa4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.288188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655552de-45c3-45a7-8047-814f7a9aa4c4 admin admin] PATH_INFO: `/v3/projects/35a0bdaa996b4280b6bf28e7ae8f541d/users/88d920ef591d46a5a9165bd44642ac6e/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.295175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1080/2153] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:13.299403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.302885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655552de-45c3-45a7-8047-814f7a9aa4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a0bdaa996b4280b6bf28e7ae8f541d, user_id=88d920ef591d46a5a9165bd44642ac6e, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.305810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655552de-45c3-45a7-8047-814f7a9aa4c4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.311394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3d746-0903-49f8-9476-b2a28a5371f4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.317742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa546a0-d881-4eba-b4ee-9ce712212677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.317999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa546a0-d881-4eba-b4ee-9ce712212677 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.318254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa546a0-d881-4eba-b4ee-9ce712212677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.318843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa546a0-d881-4eba-b4ee-9ce712212677 admin admin] PATH_INFO: `/v3/projects/22dcb880154247598045f9806f9aff5d/users/83345a36187c47ca8c9e52d3309c89e5/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.338098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1074/2154] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:13 2022] PUT /identity/v3/projects/35a0bdaa996b4280b6bf28e7ae8f541d/users/88d920ef591d46a5a9165bd44642ac6e/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:13.346645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa546a0-d881-4eba-b4ee-9ce712212677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dcb880154247598045f9806f9aff5d, user_id=83345a36187c47ca8c9e52d3309c89e5, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.349721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b280d-2a4e-4966-9a20-fb3e7e26929d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.350041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b280d-2a4e-4966-9a20-fb3e7e26929d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.350319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b280d-2a4e-4966-9a20-fb3e7e26929d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.351339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa546a0-d881-4eba-b4ee-9ce712212677 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.371308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1081/2155] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:13 2022] PUT /identity/v3/projects/22dcb880154247598045f9806f9aff5d/users/83345a36187c47ca8c9e52d3309c89e5/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:13.381809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c7b07-4630-4c11-8887-7139847c144a None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.382099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c7b07-4630-4c11-8887-7139847c144a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.382405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c7b07-4630-4c11-8887-7139847c144a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.398523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-406b280d-2a4e-4966-9a20-fb3e7e26929d None None] MFA Rules not processed for user `88d920ef591d46a5a9165bd44642ac6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:13.400076 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-406b280d-2a4e-4966-9a20-fb3e7e26929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:13.416435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-461c7b07-4630-4c11-8887-7139847c144a None None] MFA Rules not processed for user `83345a36187c47ca8c9e52d3309c89e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:13.417878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-461c7b07-4630-4c11-8887-7139847c144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:13.480804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1075/2156] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:13.485180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b280d-2a4e-4966-9a20-fb3e7e26929d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.506028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1082/2157] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:13.507183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c7b07-4630-4c11-8887-7139847c144a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.577935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b280d-2a4e-4966-9a20-fb3e7e26929d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.584543 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.585263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.585576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.586089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.586733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.592124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.607472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c7b07-4630-4c11-8887-7139847c144a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.614218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.615060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.615343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.615619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.616305 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.618447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.625552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1076/2158] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:13.629621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.656871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1083/2159] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:13.661196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.672357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:13.696420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:13.715153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161579f-9f27-4df8-aba0-81cd4089b51e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.751334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cba1e7-8b57-4f3f-881c-130547a60e0f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.756529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.757171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.757448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.757723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.758370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.760247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.785413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.789006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1084/2160] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:13.789150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.789150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.789150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.789150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.789665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.789902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.800527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb866aa-70a9-4af9-a52f-fcb357c155f0 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.807175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.808003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.808133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.808389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.809106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.810694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.818369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1077/2161] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:13 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:13.825818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1085/2162] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:13.826029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.826134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.826134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:13.833083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d5a50-4722-4bb0-b0d7-6e1e04d24300 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.838239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.838408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.838591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.838858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.839754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.841432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.855348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1078/2163] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:13.855348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.867799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1cb0f-81c6-4059-934f-07629cb85441 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.873730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.874810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.875249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.875529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] PATH_INFO: `/v3/projects/4435eee80ee84f7da6470c0295ac509c/users/d20d7c8c496e4eeea8cf9c664c7d408c/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.888286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db48675-a3e1-47d2-9fe4-43755546d9e0 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.893191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.893973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.894251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.894519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.895099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4435eee80ee84f7da6470c0295ac509c, user_id=d20d7c8c496e4eeea8cf9c664c7d408c, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.896304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:13.897848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.911941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1079/2164] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:13 2022] PUT /identity/v3/projects/4435eee80ee84f7da6470c0295ac509c/users/d20d7c8c496e4eeea8cf9c664c7d408c/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:13.912884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.957230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:13.958690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:13.969330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1086/2165] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:13.970598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:13.972723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:13.974686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c90bdc-7a4d-4263-9dbf-4db6bb842a6b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:13.980227 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:13.981083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:13.981467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:13.981877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] PATH_INFO: `/v3/projects/917f094a1bb544ad992338c207dd8530/users/f3c1ccf3d30341b1b8c55eb989abf695/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:13.989889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2368e-d4d2-4cb5-b902-7558fb393068 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.000111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.000521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917f094a1bb544ad992338c207dd8530, user_id=f3c1ccf3d30341b1b8c55eb989abf695, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.000958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.000958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.001214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] PATH_INFO: `/v3/users/970aa87c55284c32ba2c4511b0d2ba93` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.002447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=970aa87c55284c32ba2c4511b0d2ba93)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.004372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.004546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.028201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1080/2166] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:13 2022] PUT /identity/v3/projects/917f094a1bb544ad992338c207dd8530/users/f3c1ccf3d30341b1b8c55eb989abf695/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:14.037860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.067826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:14.080213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '970aa87c55284c32ba2c4511b0d2ba93'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:14.088140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:14.090550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '970aa87c55284c32ba2c4511b0d2ba93'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:14.091554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '970aa87c55284c32ba2c4511b0d2ba93'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:14.100093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '970aa87c55284c32ba2c4511b0d2ba93'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:14.113939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '970aa87c55284c32ba2c4511b0d2ba93'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:14.121857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1087/2167] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:13 2022] DELETE /identity/v3/users/970aa87c55284c32ba2c4511b0d2ba93 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:14.121857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.124838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.128407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bafd1-1daf-43a8-9136-47fd34dd55c3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.211743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a80764-a474-409e-9f0a-d429fcbc58f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.213168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a80764-a474-409e-9f0a-d429fcbc58f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.213572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a80764-a474-409e-9f0a-d429fcbc58f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.213733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a80764-a474-409e-9f0a-d429fcbc58f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.214658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a80764-a474-409e-9f0a-d429fcbc58f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.216259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a80764-a474-409e-9f0a-d429fcbc58f6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.231709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8d90d-6691-4266-8c2a-788d89afc763 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.232754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1081/2168] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:14.234365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5f7e3-c47d-4580-8452-42664b160e6c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.234793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5f7e3-c47d-4580-8452-42664b160e6c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.235081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5f7e3-c47d-4580-8452-42664b160e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.241530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.243217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.243555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.244489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.244800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.246990 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.256525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b4e5f7e3-c47d-4580-8452-42664b160e6c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:14.257734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b4e5f7e3-c47d-4580-8452-42664b160e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.262358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1088/2169] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:14.263680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.265695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.269275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1082/2170] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:14.269680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5f7e3-c47d-4580-8452-42664b160e6c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.284442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4afb-72f8-47e4-bb5a-03accfb67974 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.288084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5f7e3-c47d-4580-8452-42664b160e6c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.290086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.291374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.291667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.291949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] PATH_INFO: `/v3/projects/4435eee80ee84f7da6470c0295ac509c/users/d20d7c8c496e4eeea8cf9c664c7d408c/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.293949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4435eee80ee84f7da6470c0295ac509c, user_id=d20d7c8c496e4eeea8cf9c664c7d408c, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.294143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410bc59-cf89-476f-b3db-30e78c48ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.295029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410bc59-cf89-476f-b3db-30e78c48ec1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.295477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410bc59-cf89-476f-b3db-30e78c48ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.295879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410bc59-cf89-476f-b3db-30e78c48ec1b admin admin] PATH_INFO: `/v3/projects/917f094a1bb544ad992338c207dd8530/users/f3c1ccf3d30341b1b8c55eb989abf695/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.296651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.298061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410bc59-cf89-476f-b3db-30e78c48ec1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=917f094a1bb544ad992338c207dd8530, user_id=f3c1ccf3d30341b1b8c55eb989abf695, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.300914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410bc59-cf89-476f-b3db-30e78c48ec1b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.308384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1089/2171] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:14 2022] PUT /identity/v3/projects/4435eee80ee84f7da6470c0295ac509c/users/d20d7c8c496e4eeea8cf9c664c7d408c/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:14.309216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.311839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.315559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1083/2172] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:14 2022] PUT /identity/v3/projects/917f094a1bb544ad992338c207dd8530/users/f3c1ccf3d30341b1b8c55eb989abf695/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:14.319879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8c52c-8e76-4159-ad72-33f37ae684c3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.320298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8c52c-8e76-4159-ad72-33f37ae684c3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.320432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8c52c-8e76-4159-ad72-33f37ae684c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.354810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bcb8c52c-8e76-4159-ad72-33f37ae684c3 None None] MFA Rules not processed for user `d20d7c8c496e4eeea8cf9c664c7d408c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:14.359001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bcb8c52c-8e76-4159-ad72-33f37ae684c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.393027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99796ae-49ef-4506-85db-64c62a90feb3 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.443993 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1084/2173] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:14.445325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.445612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.445917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.457454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.458158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.458442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.458851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.464286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:14.479493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] MFA Rules not processed for user `f3c1ccf3d30341b1b8c55eb989abf695`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:14.480930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.515865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:14.538708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.540215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.550615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1090/2174] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:14.551461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.552968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.570383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1085/2175] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:14.633992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.634400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:14.657307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05de24f-28aa-4192-9ac9-4f766858eabb service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.668972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.670562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.670705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.671023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.694997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.696594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.708342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1091/2176] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:14.720594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ec324-d075-40f7-a9bb-ac2791785fd7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.725560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.726415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.726892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.727401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.739791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:14.741443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:14.751922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1086/2177] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:14.927558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:14.939765 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55f299-0119-4841-b1d2-8a537e7563f8 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:14.946365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:14.947605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:14.947894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:14.948147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:14.950556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:15.010241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.027126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd8d1-ae8e-4845-92a5-015896b40b9d service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.032234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bcce5-a25a-473a-bad9-be1c1c6f3f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.033216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bcce5-a25a-473a-bad9-be1c1c6f3f9b service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.033643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bcce5-a25a-473a-bad9-be1c1c6f3f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.034349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bcce5-a25a-473a-bad9-be1c1c6f3f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.040072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ff1bcce5-a25a-473a-bad9-be1c1c6f3f9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:15.072860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.074724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.084877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1092/2178] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:15.087095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.088518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:15.109126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47d797-558a-402d-80f1-184804ec8ce3 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.114925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.114925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.115428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.115746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] PATH_INFO: `/v3/projects/5db10dff8c774129a8d1797ce72fe258` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.117196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db10dff8c774129a8d1797ce72fe258)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.119380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.123914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bcce5-a25a-473a-bad9-be1c1c6f3f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.125455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bcce5-a25a-473a-bad9-be1c1c6f3f9b service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.137381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1087/2179] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:15.176024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:15.189191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db10dff8c774129a8d1797ce72fe258', 'request_id': 'req-fda71b10-7926-48d6-8188-ea6eef231922'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:15.199374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db10dff8c774129a8d1797ce72fe258', 'request_id': 'req-fda71b10-7926-48d6-8188-ea6eef231922'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:15.208147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db10dff8c774129a8d1797ce72fe258', 'request_id': 'req-fda71b10-7926-48d6-8188-ea6eef231922'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:15.233176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] The token cache is being invalidate because project 5db10dff8c774129a8d1797ce72fe258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:15.233591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:15.234183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fda71b10-7926-48d6-8188-ea6eef231922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:15.236870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1093/2180] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:15 2022] DELETE /identity/v3/projects/5db10dff8c774129a8d1797ce72fe258 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:15.244513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4acf2-729e-4674-8c23-84b19f725774 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.244999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4acf2-729e-4674-8c23-84b19f725774 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.245872 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4acf2-729e-4674-8c23-84b19f725774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.295748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ccc4acf2-729e-4674-8c23-84b19f725774 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:15.298475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ccc4acf2-729e-4674-8c23-84b19f725774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:15.405428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1088/2181] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:15.418763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88e844-cf08-48e4-8b44-1ceaafe8605b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.418763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88e844-cf08-48e4-8b44-1ceaafe8605b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.418763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88e844-cf08-48e4-8b44-1ceaafe8605b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.447233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-af88e844-cf08-48e4-8b44-1ceaafe8605b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:15.449412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-af88e844-cf08-48e4-8b44-1ceaafe8605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:15.459821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1094/2182] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:15.464978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4acf2-729e-4674-8c23-84b19f725774 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.472130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:15.476930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4acf2-729e-4674-8c23-84b19f725774 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.483456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.484095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.484375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.486009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.486891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:15.487217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.488929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.503295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1089/2183] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:59:15.508129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88e844-cf08-48e4-8b44-1ceaafe8605b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.522757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88e844-cf08-48e4-8b44-1ceaafe8605b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.528723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.529636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.529969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.530301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.531658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.534633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.581409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1095/2184] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:15 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:15.587293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.601028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fc04d-d26e-4b94-bdcb-b1c685caf40a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.703733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.704921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.705428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.705911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.706655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.708975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.748302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1090/2185] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:15 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:15.752107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.768248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12bc47-5c63-43c9-9a9e-261ccc8bb36b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.785481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.792207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.792494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.792770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.794279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.796045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.813687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1096/2186] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:15.817496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.830392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05120c7b-06a1-4fa3-bd32-6bb7f8bcb0f6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.836195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edb6e9-3601-4948-bc46-9e6829334920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.836997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edb6e9-3601-4948-bc46-9e6829334920 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.837281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edb6e9-3601-4948-bc46-9e6829334920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.837579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edb6e9-3601-4948-bc46-9e6829334920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.838265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edb6e9-3601-4948-bc46-9e6829334920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.840176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edb6e9-3601-4948-bc46-9e6829334920 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.853691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1091/2187] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:15.859435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:15.876758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe901f0e-77fc-4e4a-8cfc-8d2f115fbc9f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:15.882338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:15.883566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.883815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.884089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] PATH_INFO: `/v3/projects/c448b1e7eabe49799305a64b986d85eb/users/2a3739adb8b249f88dc7911ab82e48ba/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.901708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c448b1e7eabe49799305a64b986d85eb, user_id=2a3739adb8b249f88dc7911ab82e48ba, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:15.906667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:15.921579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1097/2188] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:15 2022] PUT /identity/v3/projects/c448b1e7eabe49799305a64b986d85eb/users/2a3739adb8b249f88dc7911ab82e48ba/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:15.933734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:15.934040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:15.934343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:15.973524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] MFA Rules not processed for user `2a3739adb8b249f88dc7911ab82e48ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:15.975034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:16.048294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:16.050734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:16.091875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1092/2189] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:16.168900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:16.171003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:16.178215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917411f0-d842-43fe-8e09-72fc0616c87e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:16.182740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:16.183369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:16.183652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:16.183947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:16.186468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:16.188304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82887a8-f46d-4eb2-bb86-40f243c5fa2c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:16.194544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687675-ffe5-4ae9-991d-a4115b5b0a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:16.195493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687675-ffe5-4ae9-991d-a4115b5b0a83 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:16.195951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687675-ffe5-4ae9-991d-a4115b5b0a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:16.196397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687675-ffe5-4ae9-991d-a4115b5b0a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:16.216093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687675-ffe5-4ae9-991d-a4115b5b0a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:16.217674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687675-ffe5-4ae9-991d-a4115b5b0a83 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:16.231088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1093/2190] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:16.295498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:16.297008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:16.309442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1098/2191] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:17.766424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33700a84-a416-42b5-936d-ee1665712464 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:17.766989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33700a84-a416-42b5-936d-ee1665712464 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:17.766989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33700a84-a416-42b5-936d-ee1665712464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:17.794461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-33700a84-a416-42b5-936d-ee1665712464 None None] MFA Rules not processed for user `2a3739adb8b249f88dc7911ab82e48ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:17.795766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-33700a84-a416-42b5-936d-ee1665712464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:17.805300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1094/2192] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:19.661268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:19.662828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:19.683651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:19.751718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3d99d-927a-4d80-9cf1-d0d40da6248f service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:19.757625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:19.758306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:19.758588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:19.758922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] PATH_INFO: `/v3/users/4715f245c74c4a58ac5dc27d825d6260` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:19.760280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4715f245c74c4a58ac5dc27d825d6260)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:19.762378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:19.845349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4715f245c74c4a58ac5dc27d825d6260'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:19.859182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4715f245c74c4a58ac5dc27d825d6260'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:19.860244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4715f245c74c4a58ac5dc27d825d6260'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:19.872960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4715f245c74c4a58ac5dc27d825d6260'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:19.888013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4715f245c74c4a58ac5dc27d825d6260'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:19.895172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1099/2193] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:19 2022] DELETE /identity/v3/users/4715f245c74c4a58ac5dc27d825d6260 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:19.900004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:19.900598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:19.900778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:19.923055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:19.924471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:19.979304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:20.017020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1095/2194] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:20.028162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:20.029815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:20.117460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596dfb3-ccd1-4629-90d1-b86f58367480 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:20.121702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:20.122703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:20.124082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:20.124082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:20.140221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:20.143323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:20.176871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1100/2195] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:20.556833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:20.559009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:20.582625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2093aaa-36eb-4621-abb5-b0097b56dcdb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:20.589264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:20.590086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:20.590475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:20.590943 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] PATH_INFO: `/v3/projects/08ad952b74d044dd879dbb28ab2231a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:20.593012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ad952b74d044dd879dbb28ab2231a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:20.595117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:20.683526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:20.684951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:20.706869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ad952b74d044dd879dbb28ab2231a1', 'request_id': 'req-223b92b5-0144-4504-9d11-c8751a97771c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.715462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ad952b74d044dd879dbb28ab2231a1', 'request_id': 'req-223b92b5-0144-4504-9d11-c8751a97771c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.725206 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ad952b74d044dd879dbb28ab2231a1', 'request_id': 'req-223b92b5-0144-4504-9d11-c8751a97771c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.752947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] The token cache is being invalidate because project 08ad952b74d044dd879dbb28ab2231a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:20.753423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.754202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.756313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1096/2196] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:20 2022] DELETE /identity/v3/projects/08ad952b74d044dd879dbb28ab2231a1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:20.760494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:20.762055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:20.766749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:20.788207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ecbd8-649f-4d3a-8919-6fa27458144c service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:20.860885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b92b5-0144-4504-9d11-c8751a97771c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:20.865342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:20.865853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:20.866548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:20.866834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:20.867114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] PATH_INFO: `/v3/users/5dc5a24bc6c24e75a34871837378b6ca` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:20.867420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:20.867932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:20.868340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc5a24bc6c24e75a34871837378b6ca)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:20.868472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:20.870405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:20.870547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:20.956817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc5a24bc6c24e75a34871837378b6ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.956817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc5a24bc6c24e75a34871837378b6ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.968677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc5a24bc6c24e75a34871837378b6ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.983762 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc5a24bc6c24e75a34871837378b6ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:20.987330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:20.994749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:20.997695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:21.012635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1101/2197] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:21.015319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc5a24bc6c24e75a34871837378b6ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.022277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1097/2198] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:20 2022] DELETE /identity/v3/users/5dc5a24bc6c24e75a34871837378b6ca => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:21.375502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:21.390066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd77fa2-86bb-4982-bb92-24a570ab8eca service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:21.487992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:21.489533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:21.489979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:21.490622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] PATH_INFO: `/v3/projects/552178b304a64ee0b1b2890bf58d866d` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:21.492254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] RBAC: Authorizing `identity:delete_project(project_id=552178b304a64ee0b1b2890bf58d866d)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:21.505152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:21.586715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552178b304a64ee0b1b2890bf58d866d', 'request_id': 'req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.590046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:21.600913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552178b304a64ee0b1b2890bf58d866d', 'request_id': 'req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.612293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552178b304a64ee0b1b2890bf58d866d', 'request_id': 'req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.631765 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] The token cache is being invalidate because project 552178b304a64ee0b1b2890bf58d866d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:21.632502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.635121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.645577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1102/2199] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:21 2022] DELETE /identity/v3/projects/552178b304a64ee0b1b2890bf58d866d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:21.649490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:21.651272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:21.667893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c283df-2a12-4e9d-8a61-941f94eea7eb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:21.770371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:21.771234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:21.771712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:21.772322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] PATH_INFO: `/v3/users/27386cf46d364224855efa2a3e889c9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:21.774159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] RBAC: Authorizing `identity:delete_user(user_id=27386cf46d364224855efa2a3e889c9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:21.776610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:21.856248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27386cf46d364224855efa2a3e889c9a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.857096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27386cf46d364224855efa2a3e889c9a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.867951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27386cf46d364224855efa2a3e889c9a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.905798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27386cf46d364224855efa2a3e889c9a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.922080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83ec5d01-706e-4278-ac9a-d9c7e1bd981c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27386cf46d364224855efa2a3e889c9a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:21.931085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1098/2200] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:21 2022] DELETE /identity/v3/users/27386cf46d364224855efa2a3e889c9a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:22.585618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:22.587146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:22.601542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0ae3-42d4-48d8-a8af-7fde84d5843d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:22.690080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:22.690900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:22.691251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:22.691595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] PATH_INFO: `/v3/projects/cd7ccbd1083846ccb0db0e32954e925f` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:22.692820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7ccbd1083846ccb0db0e32954e925f)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:22.695342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:22.784972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7ccbd1083846ccb0db0e32954e925f', 'request_id': 'req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:22.796509 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7ccbd1083846ccb0db0e32954e925f', 'request_id': 'req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:22.805942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7ccbd1083846ccb0db0e32954e925f', 'request_id': 'req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:22.819634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] The token cache is being invalidate because project cd7ccbd1083846ccb0db0e32954e925f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:22.819974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:22.820642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8ceffc9c-c0d4-49bc-8b5f-3f17aa3067e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:22.822954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1103/2201] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:22 2022] DELETE /identity/v3/projects/cd7ccbd1083846ccb0db0e32954e925f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:22.828714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db03c6-5d27-47e1-921a-468896a5ae82 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:22.829284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db03c6-5d27-47e1-921a-468896a5ae82 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:22.829284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db03c6-5d27-47e1-921a-468896a5ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:22.850824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-53db03c6-5d27-47e1-921a-468896a5ae82 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:22.852028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-53db03c6-5d27-47e1-921a-468896a5ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:22.928233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1099/2202] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:22.937000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a946-f5ce-411a-962f-1243e23e76cc None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:22.937288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a946-f5ce-411a-962f-1243e23e76cc None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:22.937565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a946-f5ce-411a-962f-1243e23e76cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:22.969646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a687a946-f5ce-411a-962f-1243e23e76cc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:22.972684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a687a946-f5ce-411a-962f-1243e23e76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:22.982408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1104/2203] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:22.987006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db03c6-5d27-47e1-921a-468896a5ae82 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:22.995274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:23.002900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db03c6-5d27-47e1-921a-468896a5ae82 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:23.014625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:23.017273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.017783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.018067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.018894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:23.019260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:23.020939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:23.037827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1100/2204] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:59:23.042371 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687a946-f5ce-411a-962f-1243e23e76cc None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:23.051737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687a946-f5ce-411a-962f-1243e23e76cc None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:23.056518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:23.057139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.057404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.057676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.058349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:23.060595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:23.092787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1105/2205] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:23 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:23.093998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:23.105211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3f3a0-f4f0-423f-a7ce-8d26e2f22612 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:23.240235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:23.240531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.240821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.241099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.242747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:23.244889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:23.284829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1101/2206] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:23 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:23.299469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:23.314735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68567f-a420-43a8-969d-365951ee9326 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:23.320052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:23.320917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.321248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.321529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.322297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:23.323979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:23.335979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1106/2207] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:23.351575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:23.369497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d43119-a483-4b19-88a1-945dfd0f9041 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:23.375900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4d507-f608-4497-9c90-331b29bbb64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:23.377103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d507-f608-4497-9c90-331b29bbb64d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.377857 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d507-f608-4497-9c90-331b29bbb64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.378340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d507-f608-4497-9c90-331b29bbb64d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.379367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4d507-f608-4497-9c90-331b29bbb64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:23.381183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4d507-f608-4497-9c90-331b29bbb64d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:23.395669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1102/2208] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:23.400039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:23.450794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7022e-566e-457a-a1eb-157347195e03 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:23.465698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:23.467301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.470243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.471578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] PATH_INFO: `/v3/projects/2c98c3a5bac54d78aa4480f6317b4984/users/3e802c23f70b4ac39bac6d53a6901669/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.511525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c98c3a5bac54d78aa4480f6317b4984, user_id=3e802c23f70b4ac39bac6d53a6901669, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:23.514914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:23.565868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1107/2209] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:23 2022] PUT /identity/v3/projects/2c98c3a5bac54d78aa4480f6317b4984/users/3e802c23f70b4ac39bac6d53a6901669/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:23.571075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.571387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.571665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.618360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] MFA Rules not processed for user `3e802c23f70b4ac39bac6d53a6901669`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:23.619838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:23.726980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1103/2210] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:23.823886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:23.827169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:23.834786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:23.936397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8aa73-38a1-4848-9b7b-f03166367179 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:23.941822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e83f0-a27f-4025-a58a-226d8160add6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:23.942682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e83f0-a27f-4025-a58a-226d8160add6 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:23.950354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e83f0-a27f-4025-a58a-226d8160add6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:23.950354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e83f0-a27f-4025-a58a-226d8160add6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:23.960012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e83f0-a27f-4025-a58a-226d8160add6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:23.961444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e83f0-a27f-4025-a58a-226d8160add6 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:23.973936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1108/2211] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:23.997325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:23.999368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:24.064728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:24.071361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a3654-270f-4ff0-88ae-5a3c2f863eaf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:24.077375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:24.078056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:24.078343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:24.078619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] PATH_INFO: `/v3/users/da31064c39b346f9b16d6f9b6fcc3e46` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:24.079757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] RBAC: Authorizing `identity:delete_user(user_id=da31064c39b346f9b16d6f9b6fcc3e46)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:24.081684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:24.143715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da31064c39b346f9b16d6f9b6fcc3e46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.144827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da31064c39b346f9b16d6f9b6fcc3e46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.159202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da31064c39b346f9b16d6f9b6fcc3e46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.168099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da31064c39b346f9b16d6f9b6fcc3e46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.183590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da31064c39b346f9b16d6f9b6fcc3e46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.191065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1104/2212] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:23 2022] DELETE /identity/v3/users/da31064c39b346f9b16d6f9b6fcc3e46 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:24.195957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:24.196238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:24.196583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:24.217728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:24.219123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:24.293414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1109/2213] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:24.304413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:24.306428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:24.388136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fdbb9-520c-44a2-9809-ed1dc961394b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:24.392301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:24.392918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:24.393196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:24.393464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:24.404534 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:24.406014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:24.417258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1105/2214] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:24.691501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:24.693865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:24.709119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394d8ed-9b01-4ee8-b499-94c68ce6c1ba None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:24.715518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:24.716233 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:24.716569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:24.716903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] PATH_INFO: `/v3/projects/55dd0abb05404c6aad89a27528d36b00` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:24.719050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55dd0abb05404c6aad89a27528d36b00)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:24.722681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:24.784656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55dd0abb05404c6aad89a27528d36b00', 'request_id': 'req-2e1d2ce7-644b-49ea-9642-d49ef912a827'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.795642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55dd0abb05404c6aad89a27528d36b00', 'request_id': 'req-2e1d2ce7-644b-49ea-9642-d49ef912a827'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.802411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55dd0abb05404c6aad89a27528d36b00', 'request_id': 'req-2e1d2ce7-644b-49ea-9642-d49ef912a827'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.815006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] The token cache is being invalidate because project 55dd0abb05404c6aad89a27528d36b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:24.815454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.816218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.818104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1110/2215] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:24 2022] DELETE /identity/v3/projects/55dd0abb05404c6aad89a27528d36b00 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:24.822122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:24.832827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120afe05-9aac-4f27-9da6-d74a837b4f18 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:24.893103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:24.893735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:24.894044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:24.894319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] PATH_INFO: `/v3/users/83345a36187c47ca8c9e52d3309c89e5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:24.895444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83345a36187c47ca8c9e52d3309c89e5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:24.897381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:24.933955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:24.971152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83345a36187c47ca8c9e52d3309c89e5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.971939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83345a36187c47ca8c9e52d3309c89e5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.989483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83345a36187c47ca8c9e52d3309c89e5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:24.999566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83345a36187c47ca8c9e52d3309c89e5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.013871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83345a36187c47ca8c9e52d3309c89e5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.021441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1106/2216] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:24 2022] DELETE /identity/v3/users/83345a36187c47ca8c9e52d3309c89e5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:25.358841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:25.360602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:25.386040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2ce7-644b-49ea-9642-d49ef912a827 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:25.433021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:25.479878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:25.480593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:25.480821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:25.481083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] PATH_INFO: `/v3/projects/22dcb880154247598045f9806f9aff5d` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:25.486012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dcb880154247598045f9806f9aff5d)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:25.488344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:25.566015 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dcb880154247598045f9806f9aff5d', 'request_id': 'req-458f8622-7ecf-489b-be55-1af699d84fe7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.575235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dcb880154247598045f9806f9aff5d', 'request_id': 'req-458f8622-7ecf-489b-be55-1af699d84fe7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.584225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dcb880154247598045f9806f9aff5d', 'request_id': 'req-458f8622-7ecf-489b-be55-1af699d84fe7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.596908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] The token cache is being invalidate because project 22dcb880154247598045f9806f9aff5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:25.597192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.597896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.603018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1111/2217] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:25 2022] DELETE /identity/v3/projects/22dcb880154247598045f9806f9aff5d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:25.604021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:25.605567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:25.621711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e05cdc-25e3-427a-a6c4-b16e3fae75a7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:25.737925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:25.739082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:25.739082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:25.739082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] PATH_INFO: `/v3/users/d20d7c8c496e4eeea8cf9c664c7d408c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:25.740650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20d7c8c496e4eeea8cf9c664c7d408c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:25.743565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:25.817863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20d7c8c496e4eeea8cf9c664c7d408c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.820635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20d7c8c496e4eeea8cf9c664c7d408c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.830062 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20d7c8c496e4eeea8cf9c664c7d408c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.839350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20d7c8c496e4eeea8cf9c664c7d408c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.860581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cce8e2d-53f4-4d27-947e-e641d077bf3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20d7c8c496e4eeea8cf9c664c7d408c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:25.871331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1107/2218] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:25 2022] DELETE /identity/v3/users/d20d7c8c496e4eeea8cf9c664c7d408c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:26.305804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:26.308252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:26.335069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f8622-7ecf-489b-be55-1af699d84fe7 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:26.440104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:26.440851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.441135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.441408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] PATH_INFO: `/v3/projects/4435eee80ee84f7da6470c0295ac509c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.442567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4435eee80ee84f7da6470c0295ac509c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:26.444466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:26.500935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4435eee80ee84f7da6470c0295ac509c', 'request_id': 'req-fb511276-4e64-4310-8b47-d0c9427d389a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:26.509674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4435eee80ee84f7da6470c0295ac509c', 'request_id': 'req-fb511276-4e64-4310-8b47-d0c9427d389a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:26.517579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4435eee80ee84f7da6470c0295ac509c', 'request_id': 'req-fb511276-4e64-4310-8b47-d0c9427d389a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:26.521219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:26.534451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] The token cache is being invalidate because project 4435eee80ee84f7da6470c0295ac509c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:26.534888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:26.536000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fb511276-4e64-4310-8b47-d0c9427d389a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:26.537921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1112/2219] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:26 2022] DELETE /identity/v3/projects/4435eee80ee84f7da6470c0295ac509c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:26.545192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eaf29-0b8c-4c72-87b8-75128c37b88a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.545192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eaf29-0b8c-4c72-87b8-75128c37b88a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.545192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eaf29-0b8c-4c72-87b8-75128c37b88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.566323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0c2eaf29-0b8c-4c72-87b8-75128c37b88a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:26.567616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c2eaf29-0b8c-4c72-87b8-75128c37b88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:26.580599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:26.637802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1108/2220] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:26.647291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47856fe9-194d-451f-8fff-663eb3ce8c97 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.647585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47856fe9-194d-451f-8fff-663eb3ce8c97 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.648125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47856fe9-194d-451f-8fff-663eb3ce8c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.669009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-47856fe9-194d-451f-8fff-663eb3ce8c97 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:26.670777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-47856fe9-194d-451f-8fff-663eb3ce8c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:26.678906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1113/2221] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:26.683397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eaf29-0b8c-4c72-87b8-75128c37b88a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:26.693979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eaf29-0b8c-4c72-87b8-75128c37b88a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:26.698970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:26.699594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.699878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.700157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.701009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:26.701159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:26.702834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:26.717059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1109/2222] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:59:26.721545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47856fe9-194d-451f-8fff-663eb3ce8c97 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:26.730705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47856fe9-194d-451f-8fff-663eb3ce8c97 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:26.735854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:26.736444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.736717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.736984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.737712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:26.739608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:26.770907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1114/2223] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:26 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:26.772269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:26.782716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c048a68-de94-4831-b0e8-7de0ae896d03 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:26.841071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:26.841620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.841944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.842224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.842864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:26.844770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:26.867081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1110/2224] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:26 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:26.871856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:26.883682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3074a12-ae53-4a1b-837d-b2945ce0bc71 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:26.888744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:26.889369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.889595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.889916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.890568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:26.892160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:26.903004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1115/2225] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:26.907919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:26.920567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b4246-db35-4fe6-8bee-2a57e95347e4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:26.925443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286cd51-d590-4e39-aec2-60fa742577bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:26.926076 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286cd51-d590-4e39-aec2-60fa742577bf admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.926353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286cd51-d590-4e39-aec2-60fa742577bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.926622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286cd51-d590-4e39-aec2-60fa742577bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.927270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286cd51-d590-4e39-aec2-60fa742577bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:26.929011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286cd51-d590-4e39-aec2-60fa742577bf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:26.941661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1111/2226] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:26.945834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:26.956157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d071b-a1db-4949-a899-d05a6fb58ed5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:26.960876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:26.961525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:26.961830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:26.962110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] PATH_INFO: `/v3/projects/d37aa4d432fc4f888fb1d8eef8c49e38/users/be16a324ba37482c98d24a2f81a0fcef/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:26.976786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37aa4d432fc4f888fb1d8eef8c49e38, user_id=be16a324ba37482c98d24a2f81a0fcef, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:26.979444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:26.993058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1116/2227] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:26 2022] PUT /identity/v3/projects/d37aa4d432fc4f888fb1d8eef8c49e38/users/be16a324ba37482c98d24a2f81a0fcef/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:27.006347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f3ed-cb6d-4773-ab3e-db6ddd4d39c8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.006810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f3ed-cb6d-4773-ab3e-db6ddd4d39c8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.007161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f3ed-cb6d-4773-ab3e-db6ddd4d39c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.036540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4811f3ed-cb6d-4773-ab3e-db6ddd4d39c8 None None] MFA Rules not processed for user `be16a324ba37482c98d24a2f81a0fcef`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:27.037753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4811f3ed-cb6d-4773-ab3e-db6ddd4d39c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:27.108349 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1112/2228] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:27.112951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:27.184610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ae5cd-e978-4422-9fdc-8836a2e5f690 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:27.189585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:27.190273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.190558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.190831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.191468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:27.193456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:27.226261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1117/2229] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:27 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:27.227500 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811f3ed-cb6d-4773-ab3e-db6ddd4d39c8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:27.239128 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811f3ed-cb6d-4773-ab3e-db6ddd4d39c8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:27.248302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:27.322991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:27.323262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.323541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.323821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.324551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:27.326850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:27.351834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1113/2230] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:27 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:27.355550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:27.366841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eafb5-9fdc-4d72-8f13-331e49d585f0 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:27.372423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:27.373043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.373462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.373749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.374722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:27.376808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:27.391428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1118/2231] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:27.395540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:27.408210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bfd2a-d701-4f0e-ad46-22ffdf2dd172 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:27.414125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:27.414799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.415080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.415417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] PATH_INFO: `/v3/projects/5a307f1a78254e1bb1ed332f39aa07b6/users/f643f9f81cad44e0beee53a44da7b111/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.434311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a307f1a78254e1bb1ed332f39aa07b6, user_id=f643f9f81cad44e0beee53a44da7b111, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:27.437212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:27.457033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1114/2232] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:27 2022] PUT /identity/v3/projects/5a307f1a78254e1bb1ed332f39aa07b6/users/f643f9f81cad44e0beee53a44da7b111/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:27.462141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:27.474696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e9f76-4f3e-4827-8509-14bfc3a9f125 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:27.563881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5913ef-f1c2-4a40-a58b-888c5c0f0fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:27.564971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5913ef-f1c2-4a40-a58b-888c5c0f0fec admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.565860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5913ef-f1c2-4a40-a58b-888c5c0f0fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.566298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5913ef-f1c2-4a40-a58b-888c5c0f0fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.567636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5913ef-f1c2-4a40-a58b-888c5c0f0fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:27.570216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5913ef-f1c2-4a40-a58b-888c5c0f0fec admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:27.573266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:27.597061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1119/2233] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:27.607687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:27.623363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f286b-a2d7-489a-b57a-4f8bc5bfafa1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:27.629297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:27.630006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.630287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.630567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] PATH_INFO: `/v3/projects/5a307f1a78254e1bb1ed332f39aa07b6/users/f643f9f81cad44e0beee53a44da7b111/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.633399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a307f1a78254e1bb1ed332f39aa07b6, user_id=f643f9f81cad44e0beee53a44da7b111, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:27.636914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:27.667677 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1115/2234] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:27 2022] PUT /identity/v3/projects/5a307f1a78254e1bb1ed332f39aa07b6/users/f643f9f81cad44e0beee53a44da7b111/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:27.678240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.678738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.679053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:27.710700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] MFA Rules not processed for user `f643f9f81cad44e0beee53a44da7b111`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:27.712039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:27.833588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1120/2235] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:27.845723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:27.847451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:27.930794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759e247-89fe-46ca-b9d6-701a43bbc206 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:27.935395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b432ba3-59c6-41bd-8ee3-458b4a928ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:27.936017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b432ba3-59c6-41bd-8ee3-458b4a928ade service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:27.936320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b432ba3-59c6-41bd-8ee3-458b4a928ade service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:27.936603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b432ba3-59c6-41bd-8ee3-458b4a928ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:28.010837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b432ba3-59c6-41bd-8ee3-458b4a928ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:28.013304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b432ba3-59c6-41bd-8ee3-458b4a928ade service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:28.027925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1116/2236] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:29.972258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:29.973797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:30.094986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66a45-d851-4581-8a94-cfbfb11d67e0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:30.110252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:30.111093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:30.111389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:30.111644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] PATH_INFO: `/v3/users/88d920ef591d46a5a9165bd44642ac6e` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:30.112871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d920ef591d46a5a9165bd44642ac6e)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:30.114859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:30.200201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d920ef591d46a5a9165bd44642ac6e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.208012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d920ef591d46a5a9165bd44642ac6e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.208826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d920ef591d46a5a9165bd44642ac6e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.217390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d920ef591d46a5a9165bd44642ac6e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.229369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d920ef591d46a5a9165bd44642ac6e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.236071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1121/2237] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:29 2022] DELETE /identity/v3/users/88d920ef591d46a5a9165bd44642ac6e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:30.241143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:30.241645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:30.242055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:30.262121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:30.263866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:30.276416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:30.330441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1117/2238] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:30.340516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:30.342694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:30.349971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:30.414125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdc790-6e36-486c-ac74-21d90a905e36 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:30.420643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:30.421886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:30.422387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:30.422935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:30.434413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:30.435878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:30.445380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1122/2239] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:30.729900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:30.731433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:30.743989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfc236-0a42-4e88-9981-00ea3d96307b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:30.749516 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:30.750262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:30.750542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:30.750806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] PATH_INFO: `/v3/projects/35a0bdaa996b4280b6bf28e7ae8f541d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:30.751933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a0bdaa996b4280b6bf28e7ae8f541d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:30.753879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:30.802034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a0bdaa996b4280b6bf28e7ae8f541d', 'request_id': 'req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.808384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a0bdaa996b4280b6bf28e7ae8f541d', 'request_id': 'req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.814540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a0bdaa996b4280b6bf28e7ae8f541d', 'request_id': 'req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.828735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] The token cache is being invalidate because project 35a0bdaa996b4280b6bf28e7ae8f541d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:30.829024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.829659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.831201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1118/2240] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:30 2022] DELETE /identity/v3/projects/35a0bdaa996b4280b6bf28e7ae8f541d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:30.835612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:30.845399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50427990-f718-4ce6-975f-6453c29702c7 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:30.902918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:30.903599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:30.903872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:30.904146 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] PATH_INFO: `/v3/users/f3c1ccf3d30341b1b8c55eb989abf695` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:30.905271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3c1ccf3d30341b1b8c55eb989abf695)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:30.907216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:30.955007 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3c1ccf3d30341b1b8c55eb989abf695'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.965117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3c1ccf3d30341b1b8c55eb989abf695'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.966137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3c1ccf3d30341b1b8c55eb989abf695'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.975389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3c1ccf3d30341b1b8c55eb989abf695'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.988610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-200a57f1-a988-42ab-aac6-79263099077b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3c1ccf3d30341b1b8c55eb989abf695'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:30.996297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1123/2241] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:30 2022] DELETE /identity/v3/users/f3c1ccf3d30341b1b8c55eb989abf695 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:31.362155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:31.363935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:31.382928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83c7c8-848f-44c9-9ccb-b61dfa4bfb1d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:31.540621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:31.541517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:31.541951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:31.542295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] PATH_INFO: `/v3/projects/917f094a1bb544ad992338c207dd8530` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:31.543600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=917f094a1bb544ad992338c207dd8530)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:31.546477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:31.665811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:31.672580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917f094a1bb544ad992338c207dd8530', 'request_id': 'req-a5235e5f-2536-4983-9321-a7454a679861'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:31.687674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917f094a1bb544ad992338c207dd8530', 'request_id': 'req-a5235e5f-2536-4983-9321-a7454a679861'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:31.694727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917f094a1bb544ad992338c207dd8530', 'request_id': 'req-a5235e5f-2536-4983-9321-a7454a679861'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:31.739107 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] The token cache is being invalidate because project 917f094a1bb544ad992338c207dd8530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:31.741195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:31.743428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5235e5f-2536-4983-9321-a7454a679861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:31.752520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1119/2242] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:31 2022] DELETE /identity/v3/projects/917f094a1bb544ad992338c207dd8530 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:31.754965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974fa7-67e7-405f-808e-eabae5402328 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:31.754965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974fa7-67e7-405f-808e-eabae5402328 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:31.754965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974fa7-67e7-405f-808e-eabae5402328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:31.781905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b9974fa7-67e7-405f-808e-eabae5402328 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:31.783188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b9974fa7-67e7-405f-808e-eabae5402328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:31.811790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:31.870060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1124/2243] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:31.879156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5356e-bf81-4917-a4d8-513fefefc8a9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:31.879156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5356e-bf81-4917-a4d8-513fefefc8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:31.879586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5356e-bf81-4917-a4d8-513fefefc8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:31.904721 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e7f5356e-bf81-4917-a4d8-513fefefc8a9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:31.905822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e7f5356e-bf81-4917-a4d8-513fefefc8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:31.915223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1120/2244] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:31.921753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9974fa7-67e7-405f-808e-eabae5402328 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:31.932211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9974fa7-67e7-405f-808e-eabae5402328 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:31.938235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:31.938632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:31.938913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:31.939174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:31.940135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:31.940135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:31.941697 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:31.956357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1125/2245] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:59:31.962085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5356e-bf81-4917-a4d8-513fefefc8a9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:31.974100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5356e-bf81-4917-a4d8-513fefefc8a9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:31.979688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:31.980256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:31.980710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:31.980994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:31.982188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:31.984172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.046525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1121/2246] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:31 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:32.046872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.066742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1b91a-cad5-4d8b-8aff-e2c288aecd6a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.149638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.150191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.150470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.150741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.151368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.153254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.174709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1126/2247] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:32 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:32.178959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.189816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824414bf-8180-41a2-a664-5987f4d2ae26 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.194357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.194973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.195253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.195526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.196166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.197796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.208226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1122/2248] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:32.212434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.223555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd383e-874e-4313-9ce8-39318857dc95 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.228241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3ddbd-ad50-4b42-bc90-1984874c3364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.228827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ddbd-ad50-4b42-bc90-1984874c3364 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.229105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ddbd-ad50-4b42-bc90-1984874c3364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.229372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ddbd-ad50-4b42-bc90-1984874c3364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.230043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3ddbd-ad50-4b42-bc90-1984874c3364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.231601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3ddbd-ad50-4b42-bc90-1984874c3364 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.241282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1127/2249] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:32.245398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.256538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673cf5f-19f4-4ad7-a936-56d0dac204c5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.261385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.262070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.262353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.262630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] PATH_INFO: `/v3/projects/88f93c0e43d84af88bb54e25461520ce/users/c8d9b5865a3040b6b310fba083ba0992/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.277978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f93c0e43d84af88bb54e25461520ce, user_id=c8d9b5865a3040b6b310fba083ba0992, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.281308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.298979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1123/2250] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:32 2022] PUT /identity/v3/projects/88f93c0e43d84af88bb54e25461520ce/users/c8d9b5865a3040b6b310fba083ba0992/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:32.311013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196c36-09ef-400f-a86d-ad0310dc12b9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.311299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196c36-09ef-400f-a86d-ad0310dc12b9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.311576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196c36-09ef-400f-a86d-ad0310dc12b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.342126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ca196c36-09ef-400f-a86d-ad0310dc12b9 None None] MFA Rules not processed for user `c8d9b5865a3040b6b310fba083ba0992`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:32.343249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ca196c36-09ef-400f-a86d-ad0310dc12b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:32.414073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1128/2251] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:32.419235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.506563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbde2e-57d7-4c37-a605-0ce37d5456ae admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.513558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d3903-e997-464b-a147-779a0064186b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.514226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d3903-e997-464b-a147-779a0064186b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.514503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d3903-e997-464b-a147-779a0064186b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.514787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d3903-e997-464b-a147-779a0064186b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.527975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d3903-e997-464b-a147-779a0064186b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.527975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d3903-e997-464b-a147-779a0064186b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.547743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1124/2252] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:32 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:32.551798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196c36-09ef-400f-a86d-ad0310dc12b9 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.564545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196c36-09ef-400f-a86d-ad0310dc12b9 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.584352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:32.626699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.627213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.627495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.627766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.628426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.630406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.652626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1129/2253] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:32 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:32.657045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d3903-e997-464b-a147-779a0064186b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.668099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d3903-e997-464b-a147-779a0064186b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.673269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.674091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.674378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.674652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.675297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.676912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.688408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1125/2254] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:32.693295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.709058 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de20128-bf4f-42db-913e-74d62eb4d2e2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.725956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.725956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.725956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.725956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] PATH_INFO: `/v3/projects/b0816d4ca05d453d8c86d85b66f29294/users/3b4dd1b46427433e9bfbf49d4befda3a/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.732426 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0816d4ca05d453d8c86d85b66f29294, user_id=3b4dd1b46427433e9bfbf49d4befda3a, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.735101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.749486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1130/2255] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:32 2022] PUT /identity/v3/projects/b0816d4ca05d453d8c86d85b66f29294/users/3b4dd1b46427433e9bfbf49d4befda3a/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:32.753285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.763849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839e977-1c7b-4396-be50-e1d8829db961 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.808229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:32.827412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccf9ca-edc7-4075-aae9-f2f38e92f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.828021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccf9ca-edc7-4075-aae9-f2f38e92f59a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.828302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccf9ca-edc7-4075-aae9-f2f38e92f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.828575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccf9ca-edc7-4075-aae9-f2f38e92f59a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.829247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccf9ca-edc7-4075-aae9-f2f38e92f59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.830878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccf9ca-edc7-4075-aae9-f2f38e92f59a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.842003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1126/2256] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:32.845986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:32.856421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951f64-e0f5-4b88-8a09-9e9442981dbb admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:32.861997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:32.862734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.863074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.863440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] PATH_INFO: `/v3/projects/b0816d4ca05d453d8c86d85b66f29294/users/3b4dd1b46427433e9bfbf49d4befda3a/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.865479 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0816d4ca05d453d8c86d85b66f29294, user_id=3b4dd1b46427433e9bfbf49d4befda3a, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:32.868143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:32.880155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1131/2257] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:32 2022] PUT /identity/v3/projects/b0816d4ca05d453d8c86d85b66f29294/users/3b4dd1b46427433e9bfbf49d4befda3a/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:32.890307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:32.890584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:32.890880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:32.921328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] MFA Rules not processed for user `3b4dd1b46427433e9bfbf49d4befda3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:32.922566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:32.989031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1127/2258] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:33.001167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:33.002947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:33.061111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b10771-10b5-4c5c-927a-5007afdf62eb admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:33.064900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:33.065540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:33.065825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:33.066099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:33.075370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:33.076770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:33.086862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1132/2259] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:33.190530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:33.191973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:33.203723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56363b-a65c-40b7-89d8-c08b5afd51d5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:33.207360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090658f-860c-44f1-a805-b408501f908f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:33.207964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090658f-860c-44f1-a805-b408501f908f service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:33.208240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090658f-860c-44f1-a805-b408501f908f service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:33.208508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090658f-860c-44f1-a805-b408501f908f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:33.267796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090658f-860c-44f1-a805-b408501f908f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:33.269253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090658f-860c-44f1-a805-b408501f908f service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:33.279868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1128/2260] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:36.036184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:36.053826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3d27a-58e5-46aa-b0c0-b6a6098903e1 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:36.058200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:36.058883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:36.059163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:36.059436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:36.061454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:36.146089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:36.147821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:36.161820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1133/2261] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:37.756958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46450fb5-86c3-4d3f-9be5-ce23280422d9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:37.758028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46450fb5-86c3-4d3f-9be5-ce23280422d9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:37.758136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46450fb5-86c3-4d3f-9be5-ce23280422d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:37.793486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-46450fb5-86c3-4d3f-9be5-ce23280422d9 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:37.793486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-46450fb5-86c3-4d3f-9be5-ce23280422d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:37.862436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1129/2262] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:59:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:37.935929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:37.937657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:37.970767 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:38.078523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97a721-7aac-4beb-8200-3e4a322f6df0 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:38.084332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:38.085241 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:38.085838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:38.086342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:38.097975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:38.101005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:38.112573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1134/2263] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 11:59:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:59:38.742639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be6070-f4c2-4e21-8197-4ff4a01205e2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:38.742957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be6070-f4c2-4e21-8197-4ff4a01205e2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:38.743337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be6070-f4c2-4e21-8197-4ff4a01205e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:38.767615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c5be6070-f4c2-4e21-8197-4ff4a01205e2 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:38.767615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5be6070-f4c2-4e21-8197-4ff4a01205e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:38.774886 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1130/2264] 10.222.0.88 () {58 vars in 967 bytes} [Mon Sep 19 11:59:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:38.791397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:38.801994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1d943-940d-4a94-9a2b-89b122044718 service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:38.807228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:38.807836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:38.808118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:38.808396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:38.816739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:38.818447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:38.828221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1135/2265] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 11:59:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:59:39.106769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0d6ec-b8d7-4029-8a36-a3f96f8cd60f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:39.107057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0d6ec-b8d7-4029-8a36-a3f96f8cd60f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:39.107370 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0d6ec-b8d7-4029-8a36-a3f96f8cd60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:39.115134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:39.129166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-82c0d6ec-b8d7-4029-8a36-a3f96f8cd60f None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:39.130453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82c0d6ec-b8d7-4029-8a36-a3f96f8cd60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:39.139262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1131/2266] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:39.150772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:39.161080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2c8b4-2087-4fe6-8f60-041199557ed6 service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:39.166465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:39.167149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:39.167520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:39.167863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:39.176650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:39.178482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:39.188374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1136/2267] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 11:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:59:39.578799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:39.579188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:39.579509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:39.600319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:39.601581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:39.608599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1132/2268] 10.222.0.88 () {58 vars in 967 bytes} [Mon Sep 19 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:39.620649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:39.631093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284d7d4-bc42-4363-80f2-03df75d6248a service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:39.636209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71129f55-27cf-4aa3-827e-81f19287b64c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:39.636840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71129f55-27cf-4aa3-827e-81f19287b64c service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:39.637116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71129f55-27cf-4aa3-827e-81f19287b64c service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:39.637387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71129f55-27cf-4aa3-827e-81f19287b64c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:39.646134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71129f55-27cf-4aa3-827e-81f19287b64c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:39.647693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71129f55-27cf-4aa3-827e-81f19287b64c service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:39.656269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1137/2269] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 11:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:59:43.108454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:43.110599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:43.197448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f7219-0a4f-47f2-b1f9-810a5d7a4690 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:43.205810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:43.207709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:43.208001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:43.208835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] PATH_INFO: `/v3/users/2a3739adb8b249f88dc7911ab82e48ba` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:43.210564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a3739adb8b249f88dc7911ab82e48ba)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:43.212610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:43.285669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a3739adb8b249f88dc7911ab82e48ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.286733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a3739adb8b249f88dc7911ab82e48ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.297821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a3739adb8b249f88dc7911ab82e48ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.326887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a3739adb8b249f88dc7911ab82e48ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.345482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a3739adb8b249f88dc7911ab82e48ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.354036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1133/2270] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:43 2022] DELETE /identity/v3/users/2a3739adb8b249f88dc7911ab82e48ba => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:43.357998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:43.358961 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:43.359413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:43.389291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:43.389947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:43.452385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1138/2271] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:43.462402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:43.463868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:43.523733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4f35-d0b9-44a5-8021-ddcef718e4ad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:43.527403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:43.528011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:43.528864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:43.529138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:43.538302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:43.539720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:43.549369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1134/2272] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:43.729032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:43.730579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:43.742882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5497e-b1bf-464f-b6bb-1a911c121fbd None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:43.748112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:43.748747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:43.749110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:43.749298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] PATH_INFO: `/v3/projects/c448b1e7eabe49799305a64b986d85eb` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:43.750448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c448b1e7eabe49799305a64b986d85eb)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:43.752422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:43.799419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c448b1e7eabe49799305a64b986d85eb', 'request_id': 'req-117a1221-b3dd-41f0-b548-2f3444d1f307'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.803091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:43.806570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c448b1e7eabe49799305a64b986d85eb', 'request_id': 'req-117a1221-b3dd-41f0-b548-2f3444d1f307'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.815219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c448b1e7eabe49799305a64b986d85eb', 'request_id': 'req-117a1221-b3dd-41f0-b548-2f3444d1f307'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.826567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] The token cache is being invalidate because project c448b1e7eabe49799305a64b986d85eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:43.826901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.827606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-117a1221-b3dd-41f0-b548-2f3444d1f307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:43.829226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1139/2273] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:43 2022] DELETE /identity/v3/projects/c448b1e7eabe49799305a64b986d85eb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:45.105781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:45.107982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:45.120437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12be3-0661-4cb1-9105-0319b640bbc4 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:45.128400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:45.185394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:45.186074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:45.186352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:45.186623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] PATH_INFO: `/v3/users/be16a324ba37482c98d24a2f81a0fcef` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:45.187729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be16a324ba37482c98d24a2f81a0fcef)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:45.189648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:45.241396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be16a324ba37482c98d24a2f81a0fcef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.242285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be16a324ba37482c98d24a2f81a0fcef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.250504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be16a324ba37482c98d24a2f81a0fcef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.260005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be16a324ba37482c98d24a2f81a0fcef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.273692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be16a324ba37482c98d24a2f81a0fcef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.295665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1135/2274] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:45 2022] DELETE /identity/v3/users/be16a324ba37482c98d24a2f81a0fcef => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:45.300640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:45.301326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:45.301644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:45.339453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:45.341283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:45.420946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1140/2275] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:45.432250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:45.434853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:45.470211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:45.472302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:45.489002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de987578-2f54-46bc-ad6c-a577f2ad4f0d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:45.494786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:45.495580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:45.495856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:45.496125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] PATH_INFO: `/v3/users/c8d9b5865a3040b6b310fba083ba0992` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:45.497295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d9b5865a3040b6b310fba083ba0992)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:45.499320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:45.530064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d0b2-92ea-4dcf-b120-d9a720044192 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:45.534439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc80500-dbe0-49e2-a029-2da85f27a07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:45.535326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80500-dbe0-49e2-a029-2da85f27a07c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:45.535482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80500-dbe0-49e2-a029-2da85f27a07c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:45.535814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80500-dbe0-49e2-a029-2da85f27a07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:45.546829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc80500-dbe0-49e2-a029-2da85f27a07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:45.548253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc80500-dbe0-49e2-a029-2da85f27a07c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:45.557063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d9b5865a3040b6b310fba083ba0992'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.558232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1136/2276] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:45.565823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d9b5865a3040b6b310fba083ba0992'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.566462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d9b5865a3040b6b310fba083ba0992'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.577884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d9b5865a3040b6b310fba083ba0992'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.592995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d9b5865a3040b6b310fba083ba0992'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:45.599738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1141/2277] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:45 2022] DELETE /identity/v3/users/c8d9b5865a3040b6b310fba083ba0992 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:45.605383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:45.606098 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:45.606569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:45.634230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:45.635744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:45.677523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:45.723075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1137/2278] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:45.736865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:45.739492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:45.818181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fcc1f-dddb-4bf3-8e64-0b2e1c2cfaa5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:45.827098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:45.828063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:45.828348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:45.828658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:45.839920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:45.841550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:45.845845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:45.855319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1142/2279] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:45.946059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:45.948412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:45.998033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f12aab-4219-4e2f-a57a-bbad1c79e050 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:46.004865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:46.005549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:46.005869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:46.006150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] PATH_INFO: `/v3/projects/d37aa4d432fc4f888fb1d8eef8c49e38` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:46.007866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37aa4d432fc4f888fb1d8eef8c49e38)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:46.011059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:46.064810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37aa4d432fc4f888fb1d8eef8c49e38', 'request_id': 'req-bd73a7ef-6adc-409c-a2aa-529751fa91b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.071377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37aa4d432fc4f888fb1d8eef8c49e38', 'request_id': 'req-bd73a7ef-6adc-409c-a2aa-529751fa91b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.078045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37aa4d432fc4f888fb1d8eef8c49e38', 'request_id': 'req-bd73a7ef-6adc-409c-a2aa-529751fa91b6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.096871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] The token cache is being invalidate because project d37aa4d432fc4f888fb1d8eef8c49e38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:46.097797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.098135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.100069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1138/2280] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:45 2022] DELETE /identity/v3/projects/d37aa4d432fc4f888fb1d8eef8c49e38 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:46.104138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:46.113606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576238-d53b-4ff7-a747-da3a06f53dd1 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:46.178031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:46.178762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:46.179131 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:46.179408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] PATH_INFO: `/v3/users/f643f9f81cad44e0beee53a44da7b111` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:46.180720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f643f9f81cad44e0beee53a44da7b111)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:46.182978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:46.225744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:46.227824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:46.243673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73a7ef-6adc-409c-a2aa-529751fa91b6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:46.245212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f643f9f81cad44e0beee53a44da7b111'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.250730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:46.251486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:46.251909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:46.252212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] PATH_INFO: `/v3/projects/88f93c0e43d84af88bb54e25461520ce` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:46.253825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f93c0e43d84af88bb54e25461520ce)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:46.255830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:46.259603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f643f9f81cad44e0beee53a44da7b111'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.260274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f643f9f81cad44e0beee53a44da7b111'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.272027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f643f9f81cad44e0beee53a44da7b111'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.287436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f643f9f81cad44e0beee53a44da7b111'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.295293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1143/2281] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:46 2022] DELETE /identity/v3/users/f643f9f81cad44e0beee53a44da7b111 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:46.330330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f93c0e43d84af88bb54e25461520ce', 'request_id': 'req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.336810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f93c0e43d84af88bb54e25461520ce', 'request_id': 'req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.344366 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f93c0e43d84af88bb54e25461520ce', 'request_id': 'req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.369893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] The token cache is being invalidate because project 88f93c0e43d84af88bb54e25461520ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:46.370519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.371297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.374780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1139/2282] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:46 2022] DELETE /identity/v3/projects/88f93c0e43d84af88bb54e25461520ce => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:46.379369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:46.381799 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:46.465813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c2d1-1595-495d-84c9-371350ebf04b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:46.498021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:46.562144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:46.563236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:46.563529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:46.563805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] PATH_INFO: `/v3/users/3b4dd1b46427433e9bfbf49d4befda3a` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:46.565201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b4dd1b46427433e9bfbf49d4befda3a)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:46.567724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:46.654580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b4dd1b46427433e9bfbf49d4befda3a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.663112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b4dd1b46427433e9bfbf49d4befda3a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.663832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b4dd1b46427433e9bfbf49d4befda3a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.673323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b4dd1b46427433e9bfbf49d4befda3a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.690991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-93701ef3-b442-4b28-8c98-0a5b98986d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b4dd1b46427433e9bfbf49d4befda3a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.700125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1144/2283] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:46 2022] DELETE /identity/v3/users/3b4dd1b46427433e9bfbf49d4befda3a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:46.721855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:46.723384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:46.738762 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2fd0-5783-4fd7-bedc-4fccd0440760 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:46.786767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:46.817300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:46.818483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:46.818483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:46.818724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] PATH_INFO: `/v3/projects/5a307f1a78254e1bb1ed332f39aa07b6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:46.819815 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a307f1a78254e1bb1ed332f39aa07b6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:46.822001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:46.919842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a307f1a78254e1bb1ed332f39aa07b6', 'request_id': 'req-2341552a-c2b2-4fc2-bc97-53a32b27206c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.928292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a307f1a78254e1bb1ed332f39aa07b6', 'request_id': 'req-2341552a-c2b2-4fc2-bc97-53a32b27206c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.936563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a307f1a78254e1bb1ed332f39aa07b6', 'request_id': 'req-2341552a-c2b2-4fc2-bc97-53a32b27206c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.964817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] The token cache is being invalidate because project 5a307f1a78254e1bb1ed332f39aa07b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:46.965397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.970283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2341552a-c2b2-4fc2-bc97-53a32b27206c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:46.972217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1140/2284] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:46 2022] DELETE /identity/v3/projects/5a307f1a78254e1bb1ed332f39aa07b6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:46.978000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625e665-d139-4f94-b146-698f3f20f95a None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:46.978370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625e665-d139-4f94-b146-698f3f20f95a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:46.978568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625e665-d139-4f94-b146-698f3f20f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.000748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b625e665-d139-4f94-b146-698f3f20f95a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:47.002182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b625e665-d139-4f94-b146-698f3f20f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.068677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1145/2285] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:47.078175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.078476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.078737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.099519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:47.100774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.109008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1141/2286] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:47.113335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625e665-d139-4f94-b146-698f3f20f95a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.123792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625e665-d139-4f94-b146-698f3f20f95a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.129467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.130193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.130552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.130829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.131544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:47.131894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.133530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.158466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1146/2287] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:59:47.158466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.161818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.164323 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.171866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b05dd7-45f1-4840-add9-17f7c2be2bea admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.178374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76ed23-0f06-44ce-b702-7deef17d8d78 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.179638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.180396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.180679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.180950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.181616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.183507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.183662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.183861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.184147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.184458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] PATH_INFO: `/v3/projects/b0816d4ca05d453d8c86d85b66f29294` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.186045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0816d4ca05d453d8c86d85b66f29294)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.187989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.223141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1147/2288] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:47.224571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.235159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:47.246331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0816d4ca05d453d8c86d85b66f29294', 'request_id': 'req-772d774c-6b91-4238-bbed-481d8d72cdc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:47.254162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0816d4ca05d453d8c86d85b66f29294', 'request_id': 'req-772d774c-6b91-4238-bbed-481d8d72cdc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:47.262264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0816d4ca05d453d8c86d85b66f29294', 'request_id': 'req-772d774c-6b91-4238-bbed-481d8d72cdc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:47.275807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] The token cache is being invalidate because project b0816d4ca05d453d8c86d85b66f29294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:47.276144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:47.276766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-772d774c-6b91-4238-bbed-481d8d72cdc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:47.279381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1142/2289] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:47 2022] DELETE /identity/v3/projects/b0816d4ca05d453d8c86d85b66f29294 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:47.285468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eb5d5-f1bc-4e12-a283-d902568f4f0b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.286021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eb5d5-f1bc-4e12-a283-d902568f4f0b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.286367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eb5d5-f1bc-4e12-a283-d902568f4f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.307396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6bbc0-1d23-4c63-942f-ec4292f1e530 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.309984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a59eb5d5-f1bc-4e12-a283-d902568f4f0b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:47.311104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a59eb5d5-f1bc-4e12-a283-d902568f4f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.364622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:47.366385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.367124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.367460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.367742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.368391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.370532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.385214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1143/2290] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:47.396651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fe962-7e73-4e15-9538-04debfe5c2a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.397027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fe962-7e73-4e15-9538-04debfe5c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.397379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fe962-7e73-4e15-9538-04debfe5c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.401854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1148/2291] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:47.407513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.409349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.424152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b81fe962-7e73-4e15-9538-04debfe5c2a0 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:47.424550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878ec0-67fd-4c47-9728-b8e215729c8f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.425466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b81fe962-7e73-4e15-9538-04debfe5c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.432843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.433546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.433936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.434808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.435535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.436263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1144/2292] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:47.437207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.441455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fe962-7e73-4e15-9538-04debfe5c2a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.453834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1149/2293] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:47.454013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fe962-7e73-4e15-9538-04debfe5c2a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.455159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.455914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.456511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.456785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.457053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.457694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:47.458026 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.459607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.465172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652eeef-22fb-4833-a615-ac5122bd4eb3 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.469948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.470689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.470972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.471244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.471893 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.472238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1145/2294] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:59:47.475339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.476560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.487679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1150/2295] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:47.489789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714816f-22cb-4b27-b3b3-1ffadd9da3e8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.493851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.494520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.495124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.495398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.495687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.496287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.498744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.504383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413526e-57ef-4640-9b2d-add66ea2082f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.508904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46788eb7-c56a-4679-b423-a83b653c5a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.509541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788eb7-c56a-4679-b423-a83b653c5a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.509848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788eb7-c56a-4679-b423-a83b653c5a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.510119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788eb7-c56a-4679-b423-a83b653c5a8f admin admin] PATH_INFO: `/v3/projects/12aa106fb9c74b03950fa5b35d35c80e/users/95cc7e8d318d415ca71e59312a3fbd68/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.526569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46788eb7-c56a-4679-b423-a83b653c5a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=12aa106fb9c74b03950fa5b35d35c80e, user_id=95cc7e8d318d415ca71e59312a3fbd68, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.528108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1146/2296] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:59:47.529421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46788eb7-c56a-4679-b423-a83b653c5a8f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.532774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.544266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1151/2297] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:47 2022] PUT /identity/v3/projects/12aa106fb9c74b03950fa5b35d35c80e/users/95cc7e8d318d415ca71e59312a3fbd68/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:47.555512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.555801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.556075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.584688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] MFA Rules not processed for user `95cc7e8d318d415ca71e59312a3fbd68`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:47.585922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.602703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c14406-cc6a-4004-9f99-fd0f0cc0828e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.646807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:47.666343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.666973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.667403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.667511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.668144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.670826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.676296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1152/2298] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:47.689937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.692772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:47.704527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1147/2299] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:47.710148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.722031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d3b2-d4c7-4c3e-b7cf-3191454b0be7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.727272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.727883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.728166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.728443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.729097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.730747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.742308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1148/2300] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:47.747202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.757100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c9c18-65b0-47a4-bf92-064398e28fbb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.761896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.763015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.763299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.763598 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.764035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255a4b3-2a93-4442-b824-f92472738fe7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.764261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.765962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.769316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aa74a-097b-4627-b601-0f5fb3591862 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a24bc4a0c4f480192d87a27a243acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.770284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa74a-097b-4627-b601-0f5fb3591862 service glance] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.770847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa74a-097b-4627-b601-0f5fb3591862 service glance] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.771037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa74a-097b-4627-b601-0f5fb3591862 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.781116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1149/2301] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:59:47.783150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:47.786254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aa74a-097b-4627-b601-0f5fb3591862 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.787685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aa74a-097b-4627-b601-0f5fb3591862 service glance] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.793347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d73bd2-24b2-452b-972b-207d5d0d081d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:47.798669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:47.799337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.799484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1153/2302] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:47.801559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.801882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] PATH_INFO: `/v3/projects/90534f58439f4384b228464efddd2cae/users/b20d0b7a58684d73807ae97c84b429c7/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.821198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90534f58439f4384b228464efddd2cae, user_id=b20d0b7a58684d73807ae97c84b429c7, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:47.825591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:47.830546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:47.851930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1150/2303] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:47 2022] PUT /identity/v3/projects/90534f58439f4384b228464efddd2cae/users/b20d0b7a58684d73807ae97c84b429c7/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:47.863661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe2fb-1b0f-4dbd-b9f6-6f9f0c01b057 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:47.863661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe2fb-1b0f-4dbd-b9f6-6f9f0c01b057 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:47.863661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe2fb-1b0f-4dbd-b9f6-6f9f0c01b057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:47.894357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0e8fe2fb-1b0f-4dbd-b9f6-6f9f0c01b057 None None] MFA Rules not processed for user `b20d0b7a58684d73807ae97c84b429c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:47.895584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0e8fe2fb-1b0f-4dbd-b9f6-6f9f0c01b057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:48.010963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1154/2304] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:48.015754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.108651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccccb7-3930-4635-b021-54c6f2e60739 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.113545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.114205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.114492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.114766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.115445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.117407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.152260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1151/2305] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:48 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:48.157915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe2fb-1b0f-4dbd-b9f6-6f9f0c01b057 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.169648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe2fb-1b0f-4dbd-b9f6-6f9f0c01b057 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.242906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.243521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.243813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.244110 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.244713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.246906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.279587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1155/2306] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:48 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:48.281163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.294570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e36c48-5f6e-4af9-afac-3a614e0a9654 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.299873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.300711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.300987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.301302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.301944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.303565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.324104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1152/2307] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:48.325389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.340229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae835d7e-fc55-433d-bdca-c8e54f78d278 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.346394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.347157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.347859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.348276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] PATH_INFO: `/v3/projects/3aa8df6de7d24042953537fd2d4c121c/users/29197a7a4f08482fa981168020dae023/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.363955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:48.391927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa8df6de7d24042953537fd2d4c121c, user_id=29197a7a4f08482fa981168020dae023, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.395092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.416348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1156/2308] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:48 2022] PUT /identity/v3/projects/3aa8df6de7d24042953537fd2d4c121c/users/29197a7a4f08482fa981168020dae023/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:48.419421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.432289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceda0ba-e8d4-4c34-9ebf-7d2af5fec7f8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.502651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58abde-2738-4f61-856b-651d83189849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.503341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58abde-2738-4f61-856b-651d83189849 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.503564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58abde-2738-4f61-856b-651d83189849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.503837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58abde-2738-4f61-856b-651d83189849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.504489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58abde-2738-4f61-856b-651d83189849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.506298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58abde-2738-4f61-856b-651d83189849 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.517285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1153/2309] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:48.521840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.532803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0431e-4257-41bf-9053-9e5731044c4d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.537735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.538559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.538847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.539242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] PATH_INFO: `/v3/projects/3aa8df6de7d24042953537fd2d4c121c/users/29197a7a4f08482fa981168020dae023/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.541626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa8df6de7d24042953537fd2d4c121c, user_id=29197a7a4f08482fa981168020dae023, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.545134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.559861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1157/2310] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:48 2022] PUT /identity/v3/projects/3aa8df6de7d24042953537fd2d4c121c/users/29197a7a4f08482fa981168020dae023/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:48.569881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.570171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.570849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.617725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] MFA Rules not processed for user `29197a7a4f08482fa981168020dae023`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:48.619855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:48.685055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:48.707221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1154/2311] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:48.719553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.721285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:48.800775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c46cd-fa7d-4918-b8e9-9a0e28558a1d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.804602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.805360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.805632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.805936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.817373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.818826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.831660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1158/2312] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:48.879292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:48.880698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:48.894245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f150f55-7039-4064-8b95-4cbed6d444c0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:48.898146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34021cdd-ca51-40e5-b072-a2faf54d2b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:48.898877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021cdd-ca51-40e5-b072-a2faf54d2b3c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:48.899147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021cdd-ca51-40e5-b072-a2faf54d2b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:48.899557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34021cdd-ca51-40e5-b072-a2faf54d2b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:48.959052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34021cdd-ca51-40e5-b072-a2faf54d2b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:48.960240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34021cdd-ca51-40e5-b072-a2faf54d2b3c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:48.969959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1155/2313] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:52.589574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:52.676909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5c5a4-a5cc-4167-8d33-3d4135527f36 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:52.683044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:52.683707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:52.684624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:52.684958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] PATH_INFO: `/v3/users/b20d0b7a58684d73807ae97c84b429c7` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:52.697988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20d0b7a58684d73807ae97c84b429c7)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:52.701004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:52.755772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20d0b7a58684d73807ae97c84b429c7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:52.763971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20d0b7a58684d73807ae97c84b429c7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:52.764576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20d0b7a58684d73807ae97c84b429c7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:52.772105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20d0b7a58684d73807ae97c84b429c7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:52.782988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:52.784210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20d0b7a58684d73807ae97c84b429c7'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:52.794987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1159/2314] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:52 2022] DELETE /identity/v3/users/b20d0b7a58684d73807ae97c84b429c7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:52.799372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:52.799842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:52.799990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:52.824068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:52.825805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:52.886795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1156/2315] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:52.897033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:52.898456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:52.960090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729448ce-680c-40db-b040-ebe3752915f3 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:52.963979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:52.964875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:52.965127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:52.966106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:52.975480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:52.976990 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:52.990079 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1160/2316] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:53.355660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:53.375586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:53.388767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fd86b-ac21-46f7-92a6-ef26d5465bb7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:53.393686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:53.394465 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:53.394800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:53.395137 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] PATH_INFO: `/v3/projects/90534f58439f4384b228464efddd2cae` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:53.396366 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90534f58439f4384b228464efddd2cae)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:53.398341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:53.449728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90534f58439f4384b228464efddd2cae', 'request_id': 'req-9ecf9729-3d61-46ce-8604-ab55c00e73a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.456171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90534f58439f4384b228464efddd2cae', 'request_id': 'req-9ecf9729-3d61-46ce-8604-ab55c00e73a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.461986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90534f58439f4384b228464efddd2cae', 'request_id': 'req-9ecf9729-3d61-46ce-8604-ab55c00e73a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.490222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] The token cache is being invalidate because project 90534f58439f4384b228464efddd2cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:53.490687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.491357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.492974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1157/2317] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:53 2022] DELETE /identity/v3/projects/90534f58439f4384b228464efddd2cae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:53.497029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:53.507171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e16f8a-7c1d-49da-9094-ef38b37977f0 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:53.565164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:53.565823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:53.566108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:53.566381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] PATH_INFO: `/v3/users/29197a7a4f08482fa981168020dae023` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:53.567500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29197a7a4f08482fa981168020dae023)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:53.569411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:53.623159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29197a7a4f08482fa981168020dae023'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.635877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29197a7a4f08482fa981168020dae023'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.636588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29197a7a4f08482fa981168020dae023'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.644240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29197a7a4f08482fa981168020dae023'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.659964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-60129953-89ba-46f8-a29e-c9fd377c725f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29197a7a4f08482fa981168020dae023'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:53.666164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1161/2318] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:53 2022] DELETE /identity/v3/users/29197a7a4f08482fa981168020dae023 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:54.103791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:54.109628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:54.135897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9729-3d61-46ce-8604-ab55c00e73a1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:54.148077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:54.221157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:54.221529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.221710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.221994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] PATH_INFO: `/v3/projects/3aa8df6de7d24042953537fd2d4c121c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.223238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa8df6de7d24042953537fd2d4c121c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:54.225436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:54.280254 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa8df6de7d24042953537fd2d4c121c', 'request_id': 'req-b69b2095-e20d-4683-99a7-7728f7d1f729'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:54.287335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa8df6de7d24042953537fd2d4c121c', 'request_id': 'req-b69b2095-e20d-4683-99a7-7728f7d1f729'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:54.303783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa8df6de7d24042953537fd2d4c121c', 'request_id': 'req-b69b2095-e20d-4683-99a7-7728f7d1f729'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:54.320133 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] The token cache is being invalidate because project 3aa8df6de7d24042953537fd2d4c121c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:54.320428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:54.321561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b69b2095-e20d-4683-99a7-7728f7d1f729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:54.322684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1158/2319] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:54 2022] DELETE /identity/v3/projects/3aa8df6de7d24042953537fd2d4c121c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:54.327912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b2c4a-2ac5-41b8-8f1b-d92a324c5263 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.328221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b2c4a-2ac5-41b8-8f1b-d92a324c5263 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.328636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b2c4a-2ac5-41b8-8f1b-d92a324c5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.352841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-307b2c4a-2ac5-41b8-8f1b-d92a324c5263 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:54.354172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-307b2c4a-2ac5-41b8-8f1b-d92a324c5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:54.375360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:54.457114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1162/2320] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:54.467152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5a4e-ac0e-4257-96a6-bba7f4770b9d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.467687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5a4e-ac0e-4257-96a6-bba7f4770b9d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.468096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5a4e-ac0e-4257-96a6-bba7f4770b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.493689 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c93a5a4e-ac0e-4257-96a6-bba7f4770b9d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:54.495039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c93a5a4e-ac0e-4257-96a6-bba7f4770b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:54.504064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1159/2321] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:54.508469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b2c4a-2ac5-41b8-8f1b-d92a324c5263 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:54.520664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b2c4a-2ac5-41b8-8f1b-d92a324c5263 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:54.525652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:54.526306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.526589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.526861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.527518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:54.528443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:54.530080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:54.548689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1163/2322] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:59:54.553371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a5a4e-ac0e-4257-96a6-bba7f4770b9d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:54.563562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a5a4e-ac0e-4257-96a6-bba7f4770b9d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:54.568429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:54.569142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.569525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.569853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.570568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:54.572541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:54.607204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1160/2323] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:54 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:54.611379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:54.621491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48d479-6ac5-4ca9-a4fc-d34870bc7af6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:54.688688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:54.689476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.689792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.690071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.690729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:54.692665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:54.721843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1164/2324] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:54 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:54.724775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:54.735880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953214a-8ae4-4951-9ae9-0819b9a2bc39 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:54.741825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:54.742947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.743233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.743507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.744226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:54.749036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:54.761255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1161/2325] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:54.766138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:54.776975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abee2fa-a6be-43c8-afa7-da83f1562fdb admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:54.782571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ebf5a-bb9a-476e-912d-c3e1b417f4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:54.783187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ebf5a-bb9a-476e-912d-c3e1b417f4ca admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.783468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ebf5a-bb9a-476e-912d-c3e1b417f4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.783740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ebf5a-bb9a-476e-912d-c3e1b417f4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.784393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ebf5a-bb9a-476e-912d-c3e1b417f4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:54.786050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ebf5a-bb9a-476e-912d-c3e1b417f4ca admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:54.801183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1165/2326] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:54.802541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:54.814159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392d3b4-d34e-4a61-8484-8831c61352df admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:54.824292 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:54.825528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.825838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.826121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] PATH_INFO: `/v3/projects/1e6ee50c43244d7d9e274b11429ddb13/users/b11603c7ff4b40a8aa142e5e9df1e99c/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.849100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6ee50c43244d7d9e274b11429ddb13, user_id=b11603c7ff4b40a8aa142e5e9df1e99c, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:54.851832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:54.866420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1162/2327] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:54 2022] PUT /identity/v3/projects/1e6ee50c43244d7d9e274b11429ddb13/users/b11603c7ff4b40a8aa142e5e9df1e99c/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:54.877118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023e24d-7044-480e-a838-194153477e78 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:54.877400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023e24d-7044-480e-a838-194153477e78 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:54.877826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023e24d-7044-480e-a838-194153477e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:54.911473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e023e24d-7044-480e-a838-194153477e78 None None] MFA Rules not processed for user `b11603c7ff4b40a8aa142e5e9df1e99c`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:54.912738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e023e24d-7044-480e-a838-194153477e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:54.997277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1166/2328] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:55.002068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.055443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:55.081668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1006f7-f248-4d8e-8edd-6b3926774498 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.086662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.087301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.087610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.087892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.089396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.090848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:55.124720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1163/2329] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:55 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:55.126576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023e24d-7044-480e-a838-194153477e78 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.139535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023e24d-7044-480e-a838-194153477e78 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.157029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:55.230078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.230643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.233235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.233454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.234145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.236181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:55.283254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1167/2330] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:55 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:55.287536 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.299407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a8aa4-c03a-4fe6-8333-fdf3009ecdc3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.304511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.306139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.306414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.306704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.307411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.309228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:55.327759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1164/2331] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:55.329260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.342800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565253fa-661e-4336-b577-817b1c6bb8ef admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.347860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f0fb-5f84-425e-981a-f965080ee07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.348703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f0fb-5f84-425e-981a-f965080ee07a admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.348847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f0fb-5f84-425e-981a-f965080ee07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.349136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f0fb-5f84-425e-981a-f965080ee07a admin admin] PATH_INFO: `/v3/projects/c5b081b921f54e5f9bbf76ca641c06f7/users/a6bb9a4f8bd84a88bf41401898408c27/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.368081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f0fb-5f84-425e-981a-f965080ee07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b081b921f54e5f9bbf76ca641c06f7, user_id=a6bb9a4f8bd84a88bf41401898408c27, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.372113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f0fb-5f84-425e-981a-f965080ee07a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:55.396335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1168/2332] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:55 2022] PUT /identity/v3/projects/c5b081b921f54e5f9bbf76ca641c06f7/users/a6bb9a4f8bd84a88bf41401898408c27/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:55.400568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.412089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac929-cf11-4df1-9176-91eea41e1f8f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.489184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.490080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.490401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.490674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.491331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.494060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:55.502703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.503016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.503292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.510592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1165/2333] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:55.514482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.529906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0b97-ce1e-4218-a71c-f327afdbd62b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.531486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] MFA Rules not processed for user `11af9e7e3fed4bea9187dee0ebf27551`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:55.532743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:55.535382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc39506-a0df-48e8-8847-f6bf335daff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.536340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc39506-a0df-48e8-8847-f6bf335daff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.536842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc39506-a0df-48e8-8847-f6bf335daff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.537259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc39506-a0df-48e8-8847-f6bf335daff6 admin admin] PATH_INFO: `/v3/projects/c5b081b921f54e5f9bbf76ca641c06f7/users/a6bb9a4f8bd84a88bf41401898408c27/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.540011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc39506-a0df-48e8-8847-f6bf335daff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b081b921f54e5f9bbf76ca641c06f7, user_id=a6bb9a4f8bd84a88bf41401898408c27, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.542849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc39506-a0df-48e8-8847-f6bf335daff6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:55.555728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1166/2334] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:55 2022] PUT /identity/v3/projects/c5b081b921f54e5f9bbf76ca641c06f7/users/a6bb9a4f8bd84a88bf41401898408c27/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:55.567731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b3cd9-7daf-4b15-bc41-ca1852e6f6eb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.568172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b3cd9-7daf-4b15-bc41-ca1852e6f6eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.568457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b3cd9-7daf-4b15-bc41-ca1852e6f6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.599913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9d3b3cd9-7daf-4b15-bc41-ca1852e6f6eb None None] MFA Rules not processed for user `a6bb9a4f8bd84a88bf41401898408c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:55.601146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9d3b3cd9-7daf-4b15-bc41-ca1852e6f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:55.619866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1169/2335] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:59:55.685792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1167/2336] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:55.697091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.698632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:55.766881 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453c0b7-50dd-42a4-815d-517dab152fa7 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.822851 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:55.837839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adc36c-8c12-47a6-a35b-f930314a90af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.838465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc36c-8c12-47a6-a35b-f930314a90af service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.838745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc36c-8c12-47a6-a35b-f930314a90af service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.846204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc36c-8c12-47a6-a35b-f930314a90af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.858351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adc36c-8c12-47a6-a35b-f930314a90af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.859773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adc36c-8c12-47a6-a35b-f930314a90af service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:55.874604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1170/2337] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:55.904352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b3cd9-7daf-4b15-bc41-ca1852e6f6eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:55.930671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:55.969235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b3cd9-7daf-4b15-bc41-ca1852e6f6eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:55.973756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:55.974404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:55.974680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:55.975051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] PATH_INFO: `/v3/users/b11603c7ff4b40a8aa142e5e9df1e99c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:55.976110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11603c7ff4b40a8aa142e5e9df1e99c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:55.978071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:56.029495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11603c7ff4b40a8aa142e5e9df1e99c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.030195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11603c7ff4b40a8aa142e5e9df1e99c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.039487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11603c7ff4b40a8aa142e5e9df1e99c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.047136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11603c7ff4b40a8aa142e5e9df1e99c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.058942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11603c7ff4b40a8aa142e5e9df1e99c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.064772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1168/2338] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:55 2022] DELETE /identity/v3/users/b11603c7ff4b40a8aa142e5e9df1e99c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:56.069202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:56.069489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:56.069784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:56.091717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:56.092966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:56.154037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1171/2339] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:56.163952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:56.165372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:56.226038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb3a02-ab14-4be6-a00e-67321ad33ce5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:56.229700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:56.230337 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:56.230621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:56.230944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:56.240614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:56.242072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:56.253058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1169/2340] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:56.612396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:56.623790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:56.661386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1cf24-8ebe-4629-9512-2796a9dfe410 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:56.670134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:56.670888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:56.671198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:56.671511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] PATH_INFO: `/v3/projects/1e6ee50c43244d7d9e274b11429ddb13` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:56.673356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6ee50c43244d7d9e274b11429ddb13)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:56.676708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:56.732061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6ee50c43244d7d9e274b11429ddb13', 'request_id': 'req-3689c9c9-df07-4e47-b64b-638c3c25fb91'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.740074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6ee50c43244d7d9e274b11429ddb13', 'request_id': 'req-3689c9c9-df07-4e47-b64b-638c3c25fb91'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.748339 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6ee50c43244d7d9e274b11429ddb13', 'request_id': 'req-3689c9c9-df07-4e47-b64b-638c3c25fb91'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.760483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] The token cache is being invalidate because project 1e6ee50c43244d7d9e274b11429ddb13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:56.760865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.761433 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.763475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1172/2341] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:56 2022] DELETE /identity/v3/projects/1e6ee50c43244d7d9e274b11429ddb13 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:56.768602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:56.778802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b459ca-df79-414b-90b3-c9a4f46b0240 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:56.824251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:56.842561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:56.843348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:56.843797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:56.844295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] PATH_INFO: `/v3/users/a6bb9a4f8bd84a88bf41401898408c27` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:56.845421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bb9a4f8bd84a88bf41401898408c27)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:56.847386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:56.899882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bb9a4f8bd84a88bf41401898408c27'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.900698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bb9a4f8bd84a88bf41401898408c27'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.911034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bb9a4f8bd84a88bf41401898408c27'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.918538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bb9a4f8bd84a88bf41401898408c27'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.932359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cfd4f3a7-364b-4d47-9640-269353db2bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bb9a4f8bd84a88bf41401898408c27'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:56.941600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1170/2342] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:56 2022] DELETE /identity/v3/users/a6bb9a4f8bd84a88bf41401898408c27 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:57.235390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:57.236847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:57.252368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689c9c9-df07-4e47-b64b-638c3c25fb91 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:57.318403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:57.318969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.319251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.320119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] PATH_INFO: `/v3/projects/c5b081b921f54e5f9bbf76ca641c06f7` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.321286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b081b921f54e5f9bbf76ca641c06f7)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:57.325378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:57.335691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:57.379173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b081b921f54e5f9bbf76ca641c06f7', 'request_id': 'req-db5892a1-111f-4f6f-b158-550008a18179'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:57.387761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b081b921f54e5f9bbf76ca641c06f7', 'request_id': 'req-db5892a1-111f-4f6f-b158-550008a18179'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:57.394847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b081b921f54e5f9bbf76ca641c06f7', 'request_id': 'req-db5892a1-111f-4f6f-b158-550008a18179'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:57.405986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] The token cache is being invalidate because project c5b081b921f54e5f9bbf76ca641c06f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:57.406335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:57.406928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-db5892a1-111f-4f6f-b158-550008a18179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:57.408402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1173/2343] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:57 2022] DELETE /identity/v3/projects/c5b081b921f54e5f9bbf76ca641c06f7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:57.413361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70609f5-977a-446d-9e5e-967d249a1950 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.413885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70609f5-977a-446d-9e5e-967d249a1950 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.413991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70609f5-977a-446d-9e5e-967d249a1950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.434121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b70609f5-977a-446d-9e5e-967d249a1950 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:57.435303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b70609f5-977a-446d-9e5e-967d249a1950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:57.499979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1171/2344] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:57.507678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b4277-b335-4f36-a021-7df009107b45 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.507969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b4277-b335-4f36-a021-7df009107b45 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.508251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b4277-b335-4f36-a021-7df009107b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.529195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4f7b4277-b335-4f36-a021-7df009107b45 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:57.530437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4f7b4277-b335-4f36-a021-7df009107b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:57.538414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1174/2345] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:57.542731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70609f5-977a-446d-9e5e-967d249a1950 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:57.552340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70609f5-977a-446d-9e5e-967d249a1950 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:57.557071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:57.557663 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.557968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.558239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.558908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:59:57.559210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:57.560790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:57.572185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1172/2346] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 11:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:59:57.576911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b4277-b335-4f36-a021-7df009107b45 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:57.586244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b4277-b335-4f36-a021-7df009107b45 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:57.591758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:57.592352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.592628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.592896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.593531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:57.595458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:57.626235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1175/2347] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:57 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:57.630519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:57.640874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347e1bf-b707-4696-9ba2-e25c23b17552 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:57.692671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:57.720640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:57.721419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.721792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.722180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.722937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:57.725071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:57.753959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1173/2348] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:57 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:57.755158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:57.767213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95357dbf-77a2-496f-9541-9f821c42e1a7 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:57.772121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:57.772730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.773006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.773277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.774055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:57.775544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:57.786610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1176/2349] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:57.790485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:57.800706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f058f-f4ba-487b-b957-c114b6f8d7b8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:57.805655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7503d9-ee30-4d76-8e9b-4f20648052d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:57.806319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7503d9-ee30-4d76-8e9b-4f20648052d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.806584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7503d9-ee30-4d76-8e9b-4f20648052d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.806862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7503d9-ee30-4d76-8e9b-4f20648052d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.807514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7503d9-ee30-4d76-8e9b-4f20648052d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:57.809630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7503d9-ee30-4d76-8e9b-4f20648052d2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:57.825782 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1174/2350] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:57.833876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:57.844413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dbc0f-f4ba-4e30-b1b7-948ef243c637 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:57.849234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:57.849932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.850186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.850453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] PATH_INFO: `/v3/projects/49a28ccb6e5141a798993a6fe989fa8e/users/6d613b383ade46cab84bb9592cc07467/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.865362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a28ccb6e5141a798993a6fe989fa8e, user_id=6d613b383ade46cab84bb9592cc07467, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:57.868022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:57.882350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1177/2351] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:57 2022] PUT /identity/v3/projects/49a28ccb6e5141a798993a6fe989fa8e/users/6d613b383ade46cab84bb9592cc07467/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:57.893250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b14c81-945d-41b5-8dd9-94d2c71f10b5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:57.893580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b14c81-945d-41b5-8dd9-94d2c71f10b5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:57.893842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b14c81-945d-41b5-8dd9-94d2c71f10b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:57.924623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e8b14c81-945d-41b5-8dd9-94d2c71f10b5 None None] MFA Rules not processed for user `6d613b383ade46cab84bb9592cc07467`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:57.925895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e8b14c81-945d-41b5-8dd9-94d2c71f10b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:58.000671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1175/2352] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:58.004880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:58.070290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0c22d-46a9-4375-bc27-f94b385a273d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:58.075030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:58.075632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.075974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.076188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.076812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:58.078784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:58.108104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1178/2353] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 11:59:58 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:58.112621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b14c81-945d-41b5-8dd9-94d2c71f10b5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:58.124297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b14c81-945d-41b5-8dd9-94d2c71f10b5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:58.182313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:58.182829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.183108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.183377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.184068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:58.186030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:58.207216 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1176/2354] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 11:59:58 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:59:58.211271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:58.222868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f1899-aa4a-4fc1-ae42-9911577bcc40 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:58.227795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:58.228447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.228728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.228998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.229651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:58.231474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:58.245344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1179/2355] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:58.249415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:58.259310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784aa44-6ce6-4f63-9f20-c4ddaf38ae42 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:58.264484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:58.265185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.265476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.265850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] PATH_INFO: `/v3/projects/35c3db46a6ee44ef81a6b7697d0ee8d7/users/4b54a854492d4f339270a8771128d293/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.282448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c3db46a6ee44ef81a6b7697d0ee8d7, user_id=4b54a854492d4f339270a8771128d293, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:58.285144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:58.299687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1177/2356] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:58 2022] PUT /identity/v3/projects/35c3db46a6ee44ef81a6b7697d0ee8d7/users/4b54a854492d4f339270a8771128d293/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:58.303690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:58.313739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6618d23-36ab-4d49-a94d-e035b40028ce admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:58.317646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:58.374845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9594d8b-92f7-4557-b141-6bf2a5ddb908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:58.375370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9594d8b-92f7-4557-b141-6bf2a5ddb908 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.375652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9594d8b-92f7-4557-b141-6bf2a5ddb908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.375917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9594d8b-92f7-4557-b141-6bf2a5ddb908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.376600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9594d8b-92f7-4557-b141-6bf2a5ddb908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:58.378208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9594d8b-92f7-4557-b141-6bf2a5ddb908 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:58.388466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1180/2357] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 11:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:59:58.392710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:58.403682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8fe0b-90a8-4bcb-9a59-7ba02802e51e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:58.408413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:58.409072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.409354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.409626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] PATH_INFO: `/v3/projects/35c3db46a6ee44ef81a6b7697d0ee8d7/users/4b54a854492d4f339270a8771128d293/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.411645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c3db46a6ee44ef81a6b7697d0ee8d7, user_id=4b54a854492d4f339270a8771128d293, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:58.414682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:58.426908 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1178/2358] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 11:59:58 2022] PUT /identity/v3/projects/35c3db46a6ee44ef81a6b7697d0ee8d7/users/4b54a854492d4f339270a8771128d293/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:58.438373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378eadd-ad87-4cd1-9135-aa1a20c695da None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.438661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378eadd-ad87-4cd1-9135-aa1a20c695da None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.438935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378eadd-ad87-4cd1-9135-aa1a20c695da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.468789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0378eadd-ad87-4cd1-9135-aa1a20c695da None None] MFA Rules not processed for user `4b54a854492d4f339270a8771128d293`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:58.470076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0378eadd-ad87-4cd1-9135-aa1a20c695da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:58.538880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1181/2359] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:58.549678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:58.551390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:58.567161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:58.613420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e6c34-539a-4751-8204-0739d4e1340a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:58.617024 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98f3de-c1db-4e83-b8f9-5ac6f96e8ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:58.617627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f3de-c1db-4e83-b8f9-5ac6f96e8ddc service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:58.617947 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f3de-c1db-4e83-b8f9-5ac6f96e8ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:58.618221 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f3de-c1db-4e83-b8f9-5ac6f96e8ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:58.628396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98f3de-c1db-4e83-b8f9-5ac6f96e8ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:58.629868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98f3de-c1db-4e83-b8f9-5ac6f96e8ddc service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:58.641404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1179/2360] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:59:59.085995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378eadd-ad87-4cd1-9135-aa1a20c695da None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:59.163112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378eadd-ad87-4cd1-9135-aa1a20c695da None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:59.169182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:59.169872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:59.170273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:59.170712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] PATH_INFO: `/v3/users/6d613b383ade46cab84bb9592cc07467` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:59.171948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d613b383ade46cab84bb9592cc07467)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:59.174090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:59.210456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:59:59.225440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d613b383ade46cab84bb9592cc07467'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.234338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d613b383ade46cab84bb9592cc07467'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.235106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d613b383ade46cab84bb9592cc07467'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.242682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d613b383ade46cab84bb9592cc07467'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.255166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d613b383ade46cab84bb9592cc07467'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.261573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1182/2361] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:59 2022] DELETE /identity/v3/users/6d613b383ade46cab84bb9592cc07467 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:59:59.266104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:59.266391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:59.266664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:59.286398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:59:59.287638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:59.364090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1180/2362] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 11:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:59:59.374418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:59.375840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:59.442506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632191-aba8-41c7-a302-c80431d210a8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:59.446237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:59.446866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:59.447166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:59.447463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:59.457690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:59.459125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:59.468795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1183/2363] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 11:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:59:59.755859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:59.757409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:59:59.777731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e3fb5-ce13-4430-a1e0-3db5181fc9b3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:59.782690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:59.783334 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:59.783615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:59.783887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] PATH_INFO: `/v3/projects/49a28ccb6e5141a798993a6fe989fa8e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:59.786659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a28ccb6e5141a798993a6fe989fa8e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:59.788569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:59:59.851233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a28ccb6e5141a798993a6fe989fa8e', 'request_id': 'req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.857946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a28ccb6e5141a798993a6fe989fa8e', 'request_id': 'req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.864126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a28ccb6e5141a798993a6fe989fa8e', 'request_id': 'req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.877243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] The token cache is being invalidate because project 49a28ccb6e5141a798993a6fe989fa8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:59:59.877524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.878156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:59:59.879636 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1181/2364] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 11:59:59 2022] DELETE /identity/v3/projects/49a28ccb6e5141a798993a6fe989fa8e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:59:59.883679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:59:59.894497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661c9bf-6e3f-4ffe-b057-157b04b56486 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:59:59.951304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:59:59.951928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:59:59.952204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:59:59.952492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] PATH_INFO: `/v3/users/4b54a854492d4f339270a8771128d293` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:59:59.953700 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b54a854492d4f339270a8771128d293)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:59:59.955683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:00.007171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b54a854492d4f339270a8771128d293'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.010748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:00.015596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b54a854492d4f339270a8771128d293'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.016623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b54a854492d4f339270a8771128d293'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.024989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b54a854492d4f339270a8771128d293'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.040102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-f072dcd4-4cf3-457d-bc11-a1f9374884ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b54a854492d4f339270a8771128d293'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.047870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1184/2365] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 11:59:59 2022] DELETE /identity/v3/users/4b54a854492d4f339270a8771128d293 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:00.454466 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:00.456093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:00.483108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef745c-e35b-462f-8bf6-a19ec97d5ae5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:00.518845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:00.544530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:00.545193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:00.545470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:00.545745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] PATH_INFO: `/v3/projects/35c3db46a6ee44ef81a6b7697d0ee8d7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:00.546897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c3db46a6ee44ef81a6b7697d0ee8d7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:00.548789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:00.598998 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c3db46a6ee44ef81a6b7697d0ee8d7', 'request_id': 'req-1717ea54-9ba4-4523-a0a2-bedda52be227'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.605282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c3db46a6ee44ef81a6b7697d0ee8d7', 'request_id': 'req-1717ea54-9ba4-4523-a0a2-bedda52be227'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.611074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c3db46a6ee44ef81a6b7697d0ee8d7', 'request_id': 'req-1717ea54-9ba4-4523-a0a2-bedda52be227'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.623791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] The token cache is being invalidate because project 35c3db46a6ee44ef81a6b7697d0ee8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:00:00.624203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.625070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1717ea54-9ba4-4523-a0a2-bedda52be227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:00.627481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1182/2366] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:00:00 2022] DELETE /identity/v3/projects/35c3db46a6ee44ef81a6b7697d0ee8d7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:00:00.633133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09ad7-a73f-4d9c-bace-29b67851447c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:00.633666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09ad7-a73f-4d9c-bace-29b67851447c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:00.633954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09ad7-a73f-4d9c-bace-29b67851447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:00.657804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-98f09ad7-a73f-4d9c-bace-29b67851447c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:00.659130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-98f09ad7-a73f-4d9c-bace-29b67851447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:00.721107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1185/2367] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:00:00.728618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdb3f3-d8ec-470a-99d4-3a2daf023fee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:00.728978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdb3f3-d8ec-470a-99d4-3a2daf023fee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:00.729326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdb3f3-d8ec-470a-99d4-3a2daf023fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:00.749025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-76fdb3f3-d8ec-470a-99d4-3a2daf023fee None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:00.750296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-76fdb3f3-d8ec-470a-99d4-3a2daf023fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:00.758138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1183/2368] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:00.762992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f09ad7-a73f-4d9c-bace-29b67851447c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:00.772964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f09ad7-a73f-4d9c-bace-29b67851447c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:00.778466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:00.779108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:00.779384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:00.779650 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:00.780361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:00:00.780680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:00.782330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:00.794688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1186/2369] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:00:00.798902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdb3f3-d8ec-470a-99d4-3a2daf023fee None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:00.808198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdb3f3-d8ec-470a-99d4-3a2daf023fee None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:00.813540 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:00.814175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:00.814454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:00.814729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:00.815375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:00.817609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:00.855404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1184/2370] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:00:00 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:00.859557 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:00.869750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9c545-9d2a-48e0-bae1-c07eae97b799 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:00.942480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:00.943225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:00.943691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:00.944112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:00.944649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:00.946706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:00.973038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1187/2371] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:00:00 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:00.978430 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:00.991486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233faaad-51b7-4664-a03d-e15bcbf4f2de admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.003194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.004001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.004330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.004702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.005490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.007398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.024063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1185/2372] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:01.024459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.035619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e71a8-6b44-4dcc-a2aa-0ef7ca42c84e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.041417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f3247-29ea-4d8c-8afb-1adbfa2a271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.042077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f3247-29ea-4d8c-8afb-1adbfa2a271b admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.042461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f3247-29ea-4d8c-8afb-1adbfa2a271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.042726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f3247-29ea-4d8c-8afb-1adbfa2a271b admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.043447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f3247-29ea-4d8c-8afb-1adbfa2a271b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.044963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f3247-29ea-4d8c-8afb-1adbfa2a271b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.061262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1188/2373] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:00:01.063731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.074276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592a0c5-54a3-47aa-9d51-f5777017ecc8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.079482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.080175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.080456 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.080730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] PATH_INFO: `/v3/projects/19969ab4baef4aef91c116a3e4cb041e/users/2908d230d666480486b8cb283cf91c1e/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.096972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19969ab4baef4aef91c116a3e4cb041e, user_id=2908d230d666480486b8cb283cf91c1e, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.099723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.116539 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1186/2374] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:00:01 2022] PUT /identity/v3/projects/19969ab4baef4aef91c116a3e4cb041e/users/2908d230d666480486b8cb283cf91c1e/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:01.127093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9d0c6-f87e-4076-9b99-dcf400ea4fdf None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.127386 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9d0c6-f87e-4076-9b99-dcf400ea4fdf None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.127654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9d0c6-f87e-4076-9b99-dcf400ea4fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.157520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-64e9d0c6-f87e-4076-9b99-dcf400ea4fdf None None] MFA Rules not processed for user `2908d230d666480486b8cb283cf91c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:01.158779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-64e9d0c6-f87e-4076-9b99-dcf400ea4fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:01.188919 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:01.241518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1189/2375] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:01.246135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.336161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaf99c-5538-4dfd-a8b1-5a35cbd7ce86 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.343205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.344402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.344688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.344968 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.345661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.347852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.363571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:01.387541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1187/2376] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:00:01 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:01.391231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9d0c6-f87e-4076-9b99-dcf400ea4fdf None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.404392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9d0c6-f87e-4076-9b99-dcf400ea4fdf None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.494833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.495450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.495867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.496180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.497372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.499310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.528949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1190/2377] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:00:01 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:01.530190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.540957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a50352-7b20-43ba-ac77-4368952200b1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.545628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.546268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.546551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.546827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.547470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.549065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.559844 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1188/2378] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:01.563988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.573993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be47f4-4bbc-4ced-98bb-bfe1d49e7a18 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.579466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.580126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.580406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.580680 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] PATH_INFO: `/v3/projects/425efef243db4f5a9381d0cff858a624/users/583b17e0033e4ab79fdaa81203b188ea/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.600451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425efef243db4f5a9381d0cff858a624, user_id=583b17e0033e4ab79fdaa81203b188ea, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.603170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.617527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1191/2379] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:00:01 2022] PUT /identity/v3/projects/425efef243db4f5a9381d0cff858a624/users/583b17e0033e4ab79fdaa81203b188ea/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:01.621987 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.632343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd391b0-b06a-4af4-9480-bb65d8981070 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.702994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5704e0-a56f-4c7b-b35a-b9f0dc8745be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.703605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5704e0-a56f-4c7b-b35a-b9f0dc8745be admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.703877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5704e0-a56f-4c7b-b35a-b9f0dc8745be admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.704143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5704e0-a56f-4c7b-b35a-b9f0dc8745be admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.704778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5704e0-a56f-4c7b-b35a-b9f0dc8745be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.706625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5704e0-a56f-4c7b-b35a-b9f0dc8745be admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.717245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1189/2380] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:01.721445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.732820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e95538-da74-44e1-b75c-37f3c391baa0 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:01.738349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:01.739011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.739290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.739528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] PATH_INFO: `/v3/projects/425efef243db4f5a9381d0cff858a624/users/583b17e0033e4ab79fdaa81203b188ea/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.741532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] RBAC: Authorizing `identity:create_grant(project_id=425efef243db4f5a9381d0cff858a624, user_id=583b17e0033e4ab79fdaa81203b188ea, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:01.744349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:01.756389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1192/2381] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:00:01 2022] PUT /identity/v3/projects/425efef243db4f5a9381d0cff858a624/users/583b17e0033e4ab79fdaa81203b188ea/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:00:01.826905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594522-461c-49f4-8a8f-7cb0488a118b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:01.828114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594522-461c-49f4-8a8f-7cb0488a118b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:01.828650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594522-461c-49f4-8a8f-7cb0488a118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:01.860318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8b594522-461c-49f4-8a8f-7cb0488a118b None None] MFA Rules not processed for user `583b17e0033e4ab79fdaa81203b188ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:01.861527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8b594522-461c-49f4-8a8f-7cb0488a118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:01.943188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1190/2382] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:01.954924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:01.956813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:02.025712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40df5f-99dd-4cd3-ab70-4ce33ae8596e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:02.029593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:02.030273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:02.030567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:02.030835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:02.068443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:02.102402 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:02.103850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:02.113645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1193/2383] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:06.119385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165185f0-93e1-46e8-8b56-d19cc249ebd6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:06.119922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165185f0-93e1-46e8-8b56-d19cc249ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:06.120359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165185f0-93e1-46e8-8b56-d19cc249ebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:06.154390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-165185f0-93e1-46e8-8b56-d19cc249ebd6 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:06.156017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-165185f0-93e1-46e8-8b56-d19cc249ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:06.268715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1191/2384] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:06.283211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:06.285240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:06.423871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e45b9c-b131-4aa4-9e22-7502c1c5dc23 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:06.431997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cb739-c61f-4c40-908e-9ea1eebf1255 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:06.433059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb739-c61f-4c40-908e-9ea1eebf1255 service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:06.434277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb739-c61f-4c40-908e-9ea1eebf1255 service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:06.434277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb739-c61f-4c40-908e-9ea1eebf1255 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:06.443898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cb739-c61f-4c40-908e-9ea1eebf1255 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:06.447379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cb739-c61f-4c40-908e-9ea1eebf1255 service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:06.472831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1194/2385] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:00:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:00:06.880312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165185f0-93e1-46e8-8b56-d19cc249ebd6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:06.886081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:06.968695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165185f0-93e1-46e8-8b56-d19cc249ebd6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:06.973478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:06.974113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:06.974398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:06.974720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] PATH_INFO: `/v3/users/2908d230d666480486b8cb283cf91c1e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:06.975914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2908d230d666480486b8cb283cf91c1e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:06.977915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:07.091101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2908d230d666480486b8cb283cf91c1e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:07.091588 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2908d230d666480486b8cb283cf91c1e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:07.100863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2908d230d666480486b8cb283cf91c1e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:07.122027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2908d230d666480486b8cb283cf91c1e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:07.154757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2908d230d666480486b8cb283cf91c1e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:07.175251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1192/2386] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:00:06 2022] DELETE /identity/v3/users/2908d230d666480486b8cb283cf91c1e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:07.176844 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:07.177427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:07.177901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:07.205370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:07.207383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:07.348845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1195/2387] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:00:07.360600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:07.365365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:07.481969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce3f6-c682-4482-a702-41ac9c555c49 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:07.486217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:07.486842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:07.487124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:07.487398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:07.515680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:07.521893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:07.551322 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1193/2388] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:08.203029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:08.204970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:08.224888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1093f0-56c2-4de6-b6b9-d8faa2bffd05 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:08.230578 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:08.231236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:08.231520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:08.231921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] PATH_INFO: `/v3/projects/19969ab4baef4aef91c116a3e4cb041e` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:08.233101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19969ab4baef4aef91c116a3e4cb041e)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:08.235387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:08.384376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19969ab4baef4aef91c116a3e4cb041e', 'request_id': 'req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.402065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19969ab4baef4aef91c116a3e4cb041e', 'request_id': 'req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.404655 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:08.413619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19969ab4baef4aef91c116a3e4cb041e', 'request_id': 'req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.427713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] The token cache is being invalidate because project 19969ab4baef4aef91c116a3e4cb041e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:00:08.428150 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.428954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.434933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1196/2389] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:00:08 2022] DELETE /identity/v3/projects/19969ab4baef4aef91c116a3e4cb041e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:08.435672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:08.451373 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33855c93-a6cd-460f-9e06-173d5239e87b service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:08.480401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:08.578262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:08.578262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:08.578262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:08.578262 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] PATH_INFO: `/v3/users/583b17e0033e4ab79fdaa81203b188ea` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:08.578879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] RBAC: Authorizing `identity:delete_user(user_id=583b17e0033e4ab79fdaa81203b188ea)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:08.578879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:08.648365 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583b17e0033e4ab79fdaa81203b188ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.650450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583b17e0033e4ab79fdaa81203b188ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.663435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583b17e0033e4ab79fdaa81203b188ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.674213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583b17e0033e4ab79fdaa81203b188ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.701668 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583b17e0033e4ab79fdaa81203b188ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:08.713260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1194/2390] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:00:08 2022] DELETE /identity/v3/users/583b17e0033e4ab79fdaa81203b188ea => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:09.095682 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:09.097891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:09.124370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd876c6-4fd7-4bfc-96ad-a42f9a538ca4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:09.234081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:09.235171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:09.235649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:09.236114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] PATH_INFO: `/v3/projects/425efef243db4f5a9381d0cff858a624` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:09.237489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] RBAC: Authorizing `identity:delete_project(project_id=425efef243db4f5a9381d0cff858a624)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:09.240300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:09.307954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425efef243db4f5a9381d0cff858a624', 'request_id': 'req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:09.318010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425efef243db4f5a9381d0cff858a624', 'request_id': 'req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:09.326208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425efef243db4f5a9381d0cff858a624', 'request_id': 'req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:09.337999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] The token cache is being invalidate because project 425efef243db4f5a9381d0cff858a624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:00:09.338290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:09.338954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ecbd5a33-f8dd-468f-bf2a-99b85a5ca25b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:09.341436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1197/2391] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:00:09 2022] DELETE /identity/v3/projects/425efef243db4f5a9381d0cff858a624 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:18.706601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:19.334500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=56,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:19.357403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:19.359025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:19.372768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43551dc9-43ac-4759-9da0-e658dfe905af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:19.473813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:19.474468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:19.474975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:19.476029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] PATH_INFO: `/v3/users/3e802c23f70b4ac39bac6d53a6901669` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:19.477240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e802c23f70b4ac39bac6d53a6901669)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:19.482530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:19.537304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e802c23f70b4ac39bac6d53a6901669'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:19.538103 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e802c23f70b4ac39bac6d53a6901669'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:19.546860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e802c23f70b4ac39bac6d53a6901669'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:19.555631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e802c23f70b4ac39bac6d53a6901669'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:19.568961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e802c23f70b4ac39bac6d53a6901669'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:19.575300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1195/2392] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:00:19 2022] DELETE /identity/v3/users/3e802c23f70b4ac39bac6d53a6901669 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:00:19.579853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:19.580135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:19.580406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:19.601466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:19.602917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:19.677271 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1198/2393] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:19.687362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:19.688759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:19.745061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a943b3-60e6-4f9b-8ac1-20bfa0a5ba8a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:19.748785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0879ef-f494-4dc8-b340-dfce30dc89f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:19.749390 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0879ef-f494-4dc8-b340-dfce30dc89f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:19.749670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0879ef-f494-4dc8-b340-dfce30dc89f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:19.749972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0879ef-f494-4dc8-b340-dfce30dc89f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:19.759211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0879ef-f494-4dc8-b340-dfce30dc89f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:19.760541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0879ef-f494-4dc8-b340-dfce30dc89f4 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:19.770191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1196/2394] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:20.049714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:20.053404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:20.068125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec766b-70ed-4acb-a44c-70b7ebcd9dd3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:20.074258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:20.075034 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:20.075382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:20.075773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] PATH_INFO: `/v3/projects/2c98c3a5bac54d78aa4480f6317b4984` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:20.077175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c98c3a5bac54d78aa4480f6317b4984)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:20.079571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:20.133138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c98c3a5bac54d78aa4480f6317b4984', 'request_id': 'req-a1827882-0fe6-4300-a189-464b2f3092ae'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:20.151122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c98c3a5bac54d78aa4480f6317b4984', 'request_id': 'req-a1827882-0fe6-4300-a189-464b2f3092ae'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:20.160561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c98c3a5bac54d78aa4480f6317b4984', 'request_id': 'req-a1827882-0fe6-4300-a189-464b2f3092ae'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:20.172785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] The token cache is being invalidate because project 2c98c3a5bac54d78aa4480f6317b4984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:00:20.173083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:20.173749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a1827882-0fe6-4300-a189-464b2f3092ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:20.175449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1199/2395] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:00:20 2022] DELETE /identity/v3/projects/2c98c3a5bac54d78aa4480f6317b4984 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:27.682627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25e01e-e28d-4d7b-965f-69b8eff652eb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:27.683766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25e01e-e28d-4d7b-965f-69b8eff652eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:27.684265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25e01e-e28d-4d7b-965f-69b8eff652eb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:27.687180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1197/2396] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 12:00:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 12:00:27.694262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b48b5-a30f-41ee-92e9-bc97d3b4b9bb None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:27.695116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b48b5-a30f-41ee-92e9-bc97d3b4b9bb None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:27.695116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b48b5-a30f-41ee-92e9-bc97d3b4b9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:27.722320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-1a1b48b5-a30f-41ee-92e9-bc97d3b4b9bb None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:27.724356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1a1b48b5-a30f-41ee-92e9-bc97d3b4b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:27.827717 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1200/2397] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 12:00:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:27.841371 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25e01e-e28d-4d7b-965f-69b8eff652eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:27.843135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e25e01e-e28d-4d7b-965f-69b8eff652eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:27.879776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:27.918997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25e01e-e28d-4d7b-965f-69b8eff652eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:27.922798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11233a77-723a-4fc9-a306-038e25b7f058 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a24bc4a0c4f480192d87a27a243acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:27.923442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11233a77-723a-4fc9-a306-038e25b7f058 service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:27.923720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11233a77-723a-4fc9-a306-038e25b7f058 service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:27.923992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11233a77-723a-4fc9-a306-038e25b7f058 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:27.934809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11233a77-723a-4fc9-a306-038e25b7f058 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:27.936229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11233a77-723a-4fc9-a306-038e25b7f058 service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:27.947553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1198/2398] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:28.021240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44fa7-32bf-40dd-a0a5-5dc536750911 None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.021573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44fa7-32bf-40dd-a0a5-5dc536750911 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.021825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44fa7-32bf-40dd-a0a5-5dc536750911 None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.023318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1201/2399] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 12:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 12:00:28.029604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5c34-8ec4-4a45-be32-6f2490443888 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.030267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5c34-8ec4-4a45-be32-6f2490443888 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.031003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5c34-8ec4-4a45-be32-6f2490443888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.059122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-591b5c34-8ec4-4a45-be32-6f2490443888 None None] MFA Rules not processed for user `dee035c97cb6447382365faf6b6aca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:28.060463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-591b5c34-8ec4-4a45-be32-6f2490443888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:28.113492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1199/2400] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:00:28.125284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44fa7-32bf-40dd-a0a5-5dc536750911 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:28.127038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6ff44fa7-32bf-40dd-a0a5-5dc536750911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:28.140423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44fa7-32bf-40dd-a0a5-5dc536750911 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:28.143967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb146c4-0e6a-4859-b32a-5e2549a34955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:28.144564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb146c4-0e6a-4859-b32a-5e2549a34955 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.144843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb146c4-0e6a-4859-b32a-5e2549a34955 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.145116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb146c4-0e6a-4859-b32a-5e2549a34955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.153073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb146c4-0e6a-4859-b32a-5e2549a34955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:28.154520 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb146c4-0e6a-4859-b32a-5e2549a34955 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:28.162479 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1202/2401] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:28.247951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b317968-0b86-4734-ad73-6d09b9b34e1f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.248233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b317968-0b86-4734-ad73-6d09b9b34e1f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.248505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b317968-0b86-4734-ad73-6d09b9b34e1f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.249900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1200/2402] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 12:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 12:00:28.255519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.255811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.256092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.264353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:28.278738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:28.279991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:28.289288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1203/2403] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:28.300263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b317968-0b86-4734-ad73-6d09b9b34e1f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:28.312440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b317968-0b86-4734-ad73-6d09b9b34e1f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:28.317596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0664b8e-1959-4c42-8d6e-23b5cbcb6182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:28.318605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0664b8e-1959-4c42-8d6e-23b5cbcb6182 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.319020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0664b8e-1959-4c42-8d6e-23b5cbcb6182 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.319427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0664b8e-1959-4c42-8d6e-23b5cbcb6182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.332101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:28.333599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:28.335788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0664b8e-1959-4c42-8d6e-23b5cbcb6182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:28.338166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0664b8e-1959-4c42-8d6e-23b5cbcb6182 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:28.354016 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1201/2404] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:00:28.407529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f6323-7a08-4b1b-a0b1-264f25471c6a None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:28.412280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:28.412931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.413332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.413332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] PATH_INFO: `/v3/users/95cc7e8d318d415ca71e59312a3fbd68` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.415436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95cc7e8d318d415ca71e59312a3fbd68)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:28.417688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:28.468154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95cc7e8d318d415ca71e59312a3fbd68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:28.476797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95cc7e8d318d415ca71e59312a3fbd68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:28.477615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95cc7e8d318d415ca71e59312a3fbd68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:28.484702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95cc7e8d318d415ca71e59312a3fbd68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:28.498511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95cc7e8d318d415ca71e59312a3fbd68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:28.504667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1204/2405] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:00:28 2022] DELETE /identity/v3/users/95cc7e8d318d415ca71e59312a3fbd68 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:00:28.509159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.509444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.509714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.530085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:28.531306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:28.599368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1202/2406] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:00:28.610542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:28.611954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:28.807234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34254275-c554-4ccc-9fae-a593143368e2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:28.811003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a781745-069c-4810-bf4c-dca6d9d39cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:28.811613 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781745-069c-4810-bf4c-dca6d9d39cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:28.811871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781745-069c-4810-bf4c-dca6d9d39cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:28.812136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781745-069c-4810-bf4c-dca6d9d39cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:28.821039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a781745-069c-4810-bf4c-dca6d9d39cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:28.822481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a781745-069c-4810-bf4c-dca6d9d39cb5 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:28.832170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1205/2407] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:29.184704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.186207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:29.197855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d27a9-f53f-4089-9b21-31a505a51c1c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.203027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.203657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.203940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.204208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] PATH_INFO: `/v3/projects/12aa106fb9c74b03950fa5b35d35c80e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.205328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12aa106fb9c74b03950fa5b35d35c80e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.207269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.255487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12aa106fb9c74b03950fa5b35d35c80e', 'request_id': 'req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:29.262163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12aa106fb9c74b03950fa5b35d35c80e', 'request_id': 'req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:29.264241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:29.268924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12aa106fb9c74b03950fa5b35d35c80e', 'request_id': 'req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:29.281847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] The token cache is being invalidate because project 12aa106fb9c74b03950fa5b35d35c80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:00:29.282136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:29.282747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ca8fa3c2-ab26-4f2f-b501-b6b4c2e133f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:29.284232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1203/2408] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:00:29 2022] DELETE /identity/v3/projects/12aa106fb9c74b03950fa5b35d35c80e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:29.289428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc814b-da56-4adf-9a98-03df7719cf17 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.289710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc814b-da56-4adf-9a98-03df7719cf17 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.290015 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc814b-da56-4adf-9a98-03df7719cf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.310928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-69bc814b-da56-4adf-9a98-03df7719cf17 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:29.312061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-69bc814b-da56-4adf-9a98-03df7719cf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:29.325313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:29.376158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1206/2409] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:29.383778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3830-9881-4318-a89b-88f624539081 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.383991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3830-9881-4318-a89b-88f624539081 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.384263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3830-9881-4318-a89b-88f624539081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.409594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d20d3830-9881-4318-a89b-88f624539081 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:29.411005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d20d3830-9881-4318-a89b-88f624539081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:29.418812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1204/2410] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:29.423143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc814b-da56-4adf-9a98-03df7719cf17 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.433782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc814b-da56-4adf-9a98-03df7719cf17 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.438279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.438876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.439155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.439429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.440070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:00:29.440376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.442025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.453937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1207/2411] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:00:29.458291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3830-9881-4318-a89b-88f624539081 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.467919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3830-9881-4318-a89b-88f624539081 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.472538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.473138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.473422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.473695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.474374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.476270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.503313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1205/2412] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:00:29 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:29.507651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.518067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685146e-1e3c-4349-98e4-50df1880ddba admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.572147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.572667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.572940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.573203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.574418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.576297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.598453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1208/2413] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:00:29 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:29.602696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.613196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b64802-50c1-4500-b112-e540d91c5e43 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.617712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.618345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.618625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.618897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.619531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.621605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.631774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1206/2414] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:29.635737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.646377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6cfa-667b-4f2a-a101-78bbce461917 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.651553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4841f7-d6f9-446e-9e12-8fb92abe77ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.652189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4841f7-d6f9-446e-9e12-8fb92abe77ca admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.652461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4841f7-d6f9-446e-9e12-8fb92abe77ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.652736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4841f7-d6f9-446e-9e12-8fb92abe77ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.653377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4841f7-d6f9-446e-9e12-8fb92abe77ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.655046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4841f7-d6f9-446e-9e12-8fb92abe77ca admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.664837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1209/2415] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:29.669816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.679567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef213-7b3f-44e4-86f3-3a464ec305ff admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.684742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.685386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.685664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.685972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] PATH_INFO: `/v3/projects/fb1eada69a104728a6406561b253334a/users/786a3f2115c34e4ab30936594c361cba/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.701695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1eada69a104728a6406561b253334a, user_id=786a3f2115c34e4ab30936594c361cba, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.704426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.717843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1207/2416] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:00:29 2022] PUT /identity/v3/projects/fb1eada69a104728a6406561b253334a/users/786a3f2115c34e4ab30936594c361cba/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:29.729175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de855a0-6bc4-48dd-8751-d91ca74eb70e None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.729474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de855a0-6bc4-48dd-8751-d91ca74eb70e None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.730264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de855a0-6bc4-48dd-8751-d91ca74eb70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.758290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5de855a0-6bc4-48dd-8751-d91ca74eb70e None None] MFA Rules not processed for user `786a3f2115c34e4ab30936594c361cba`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:29.759541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5de855a0-6bc4-48dd-8751-d91ca74eb70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:29.823534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1210/2417] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:29.835506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.837205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:29.892769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7f145-9280-45ea-9cae-fa1953e2021a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:29.896353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d33fb7-d5eb-4aae-8197-0f88a9220975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:29.896949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d33fb7-d5eb-4aae-8197-0f88a9220975 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:29.897223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d33fb7-d5eb-4aae-8197-0f88a9220975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:29.897489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d33fb7-d5eb-4aae-8197-0f88a9220975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:29.905702 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d33fb7-d5eb-4aae-8197-0f88a9220975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:29.907145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d33fb7-d5eb-4aae-8197-0f88a9220975 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:29.915791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1208/2418] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:29.941690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de855a0-6bc4-48dd-8751-d91ca74eb70e None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:29.954639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:30.008333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de855a0-6bc4-48dd-8751-d91ca74eb70e None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:30.013484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:30.014172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:30.014443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:30.014714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] PATH_INFO: `/v3/users/786a3f2115c34e4ab30936594c361cba` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:30.016302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=786a3f2115c34e4ab30936594c361cba)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:30.019570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:30.067698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '786a3f2115c34e4ab30936594c361cba'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.074956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '786a3f2115c34e4ab30936594c361cba'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.075589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '786a3f2115c34e4ab30936594c361cba'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.082912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '786a3f2115c34e4ab30936594c361cba'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.095301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '786a3f2115c34e4ab30936594c361cba'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.101028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1211/2419] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:00:29 2022] DELETE /identity/v3/users/786a3f2115c34e4ab30936594c361cba => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:00:30.106445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:30.106571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:30.106846 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:30.125921 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:30.133009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:30.197358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1209/2420] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:30.208389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:30.209869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:30.323038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce6a6-67a9-4bd2-9009-46a75caefa9b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:30.327304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf18b6-bb2e-4d82-bbba-342dd9797feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:30.327841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf18b6-bb2e-4d82-bbba-342dd9797feb service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:30.328114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf18b6-bb2e-4d82-bbba-342dd9797feb service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:30.328379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf18b6-bb2e-4d82-bbba-342dd9797feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:30.337524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf18b6-bb2e-4d82-bbba-342dd9797feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:30.338951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf18b6-bb2e-4d82-bbba-342dd9797feb service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:30.348476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1212/2421] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:30.835450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:30.837242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:30.851069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba712c-fc23-4d85-bd1a-98a6b2201684 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:30.870595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:30.871275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:30.871556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:30.871852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] PATH_INFO: `/v3/projects/fb1eada69a104728a6406561b253334a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:30.882644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1eada69a104728a6406561b253334a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:30.884586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:30.910799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:30.939034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1eada69a104728a6406561b253334a', 'request_id': 'req-4b0733e9-cc17-49ff-acb1-265b56da8d1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.945755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1eada69a104728a6406561b253334a', 'request_id': 'req-4b0733e9-cc17-49ff-acb1-265b56da8d1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.952240 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1eada69a104728a6406561b253334a', 'request_id': 'req-4b0733e9-cc17-49ff-acb1-265b56da8d1a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.965345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] The token cache is being invalidate because project fb1eada69a104728a6406561b253334a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:00:30.965662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.966343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b0733e9-cc17-49ff-acb1-265b56da8d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:00:30.967767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1210/2422] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:00:30 2022] DELETE /identity/v3/projects/fb1eada69a104728a6406561b253334a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:30.972971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143601a1-32ac-4518-a2da-2347dc33c82b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:30.973477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143601a1-32ac-4518-a2da-2347dc33c82b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:30.973651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143601a1-32ac-4518-a2da-2347dc33c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:30.994654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-143601a1-32ac-4518-a2da-2347dc33c82b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:30.995864 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-143601a1-32ac-4518-a2da-2347dc33c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:31.026449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:31.058719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1213/2423] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:00:31.066488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27f26f-2a40-48ea-af7b-20bacaf7bf66 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.066779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27f26f-2a40-48ea-af7b-20bacaf7bf66 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.067060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27f26f-2a40-48ea-af7b-20bacaf7bf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.087082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6b27f26f-2a40-48ea-af7b-20bacaf7bf66 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:31.088115 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6b27f26f-2a40-48ea-af7b-20bacaf7bf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:31.096615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1211/2424] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:31.100998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143601a1-32ac-4518-a2da-2347dc33c82b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.111220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143601a1-32ac-4518-a2da-2347dc33c82b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.115714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.116308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.116585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.116853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.117489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:00:31.117816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.119383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.130988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1214/2425] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:00:31.135391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27f26f-2a40-48ea-af7b-20bacaf7bf66 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.144999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27f26f-2a40-48ea-af7b-20bacaf7bf66 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.150017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.150355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.150633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.150900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.152029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.153942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.196725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1212/2426] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:00:31 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:31.201464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.214836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8065cc3-61bd-47c3-b86e-bd2aa3d326d6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.258609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.260613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:31.293843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.294461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.294740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.295012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.295836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.297885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.329950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1215/2427] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:00:31 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:31.334763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.351119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78d15f-7ab0-419b-9a4e-6f605c2b25e4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.365675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.365675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.365675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.365675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.368013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.369656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.386387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916c61-bb56-48db-bb60-7f657698b4e2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.394710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1216/2428] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:31.396147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.403667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216967ee-e201-42c6-8664-8b10fc4b91ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.404265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216967ee-e201-42c6-8664-8b10fc4b91ff service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.404619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216967ee-e201-42c6-8664-8b10fc4b91ff service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.406039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216967ee-e201-42c6-8664-8b10fc4b91ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.407693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-216967ee-e201-42c6-8664-8b10fc4b91ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:31.410498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6131a1f-4e22-45da-aaf2-74c7e3a07e91 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.415253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.415947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.416408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.416741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.418140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.420660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.424175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216967ee-e201-42c6-8664-8b10fc4b91ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.426682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216967ee-e201-42c6-8664-8b10fc4b91ff service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.437165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1217/2429] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:31.438561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.448272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1213/2430] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:00:31.451660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b663-5b26-4eca-b724-b51b1814744f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.462909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.463707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.464029 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.464580 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] PATH_INFO: `/v3/projects/2f7dd7174903460989484c1ab233208a/users/fc84618366b045658728196980072790/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.488408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7dd7174903460989484c1ab233208a, user_id=fc84618366b045658728196980072790, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.492033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.511456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1218/2431] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:00:31 2022] PUT /identity/v3/projects/2f7dd7174903460989484c1ab233208a/users/fc84618366b045658728196980072790/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:31.516485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4350d-7692-4ab9-8b30-5041c092a7ae None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.516785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4350d-7692-4ab9-8b30-5041c092a7ae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.517067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4350d-7692-4ab9-8b30-5041c092a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.540187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b8a4350d-7692-4ab9-8b30-5041c092a7ae None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:31.541547 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b8a4350d-7692-4ab9-8b30-5041c092a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:31.614481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1214/2432] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:00:31.624829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:31.626530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:31.694003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20537e-89e7-4f8b-9ef0-24e35024ba41 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:31.697747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bcddb-790e-4b4f-bc41-3cff6fc86dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:31.698464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bcddb-790e-4b4f-bc41-3cff6fc86dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:31.699245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bcddb-790e-4b4f-bc41-3cff6fc86dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:31.699512 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bcddb-790e-4b4f-bc41-3cff6fc86dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:31.710045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bcddb-790e-4b4f-bc41-3cff6fc86dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:31.712603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bcddb-790e-4b4f-bc41-3cff6fc86dc4 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:31.733327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1219/2433] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:35.864923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7e64a-cb18-4658-9fc4-b8783e3ea061 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:35.865484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7e64a-cb18-4658-9fc4-b8783e3ea061 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:35.865484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7e64a-cb18-4658-9fc4-b8783e3ea061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:35.890962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-45c7e64a-cb18-4658-9fc4-b8783e3ea061 None None] MFA Rules not processed for user `828f876065e4413f9447efb451cb5371`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:35.892327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-45c7e64a-cb18-4658-9fc4-b8783e3ea061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:35.896950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:35.993139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1215/2434] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:36.792989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba698ec-26a0-46ef-bcd6-67fd4a5971c5 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:36.793562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba698ec-26a0-46ef-bcd6-67fd4a5971c5 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:36.793562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba698ec-26a0-46ef-bcd6-67fd4a5971c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:36.826713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:36.842347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fba698ec-26a0-46ef-bcd6-67fd4a5971c5 None None] MFA Rules not processed for user `fc84618366b045658728196980072790`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:36.844659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fba698ec-26a0-46ef-bcd6-67fd4a5971c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:36.933914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1220/2435] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:36.936259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7e64a-cb18-4658-9fc4-b8783e3ea061 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:36.959051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7e64a-cb18-4658-9fc4-b8783e3ea061 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:36.963883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:36.964806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:36.965031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:36.965297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:36.965977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:36.967893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:37.017378 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1216/2436] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:00:36 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:37.021179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba698ec-26a0-46ef-bcd6-67fd4a5971c5 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:37.037800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba698ec-26a0-46ef-bcd6-67fd4a5971c5 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:37.188738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:37.189369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:37.189559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:37.189865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:37.190525 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:37.192595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:37.240135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1221/2437] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:00:37 2022] POST /identity/v3/users => generated 295 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:00:37.244369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:37.260848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a0662-1154-4a4b-b36d-877061469d5f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:37.267537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:37.268568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:37.269462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:37.270257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:37.271168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:37.273082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:37.287179 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1217/2438] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:37.291435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:37.307072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb15e-228a-4beb-b047-a23dc336bb46 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:37.320595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9daa12-d5ff-45b4-b7c9-883dce2512a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:37.321492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9daa12-d5ff-45b4-b7c9-883dce2512a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:37.322002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9daa12-d5ff-45b4-b7c9-883dce2512a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:37.323690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9daa12-d5ff-45b4-b7c9-883dce2512a4 admin admin] PATH_INFO: `/v3/projects/dd9c395d11c54901859e6081ea0afcd9/users/789b98ddfef744239d765d3e2280aae0/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:37.349702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9daa12-d5ff-45b4-b7c9-883dce2512a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9c395d11c54901859e6081ea0afcd9, user_id=789b98ddfef744239d765d3e2280aae0, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:37.352415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9daa12-d5ff-45b4-b7c9-883dce2512a4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:37.379399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1222/2439] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:00:37 2022] PUT /identity/v3/projects/dd9c395d11c54901859e6081ea0afcd9/users/789b98ddfef744239d765d3e2280aae0/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:37.381003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:37.397335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca3250-e8f5-42ec-b11c-db7e1a0d2f1c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:37.437038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:37.437313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:37.437587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:37.486182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:37.487752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:37.497388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:37.497972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:37.498451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:37.498770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:37.499285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:37.501099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:37.517789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1218/2440] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:00:37.518809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:37.535897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497126-8f9e-48c8-878f-8582d173cb8c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:37.541796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9ae75-3ec2-499e-9f30-6b8ae58255a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:37.545267 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ae75-3ec2-499e-9f30-6b8ae58255a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:37.545468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ae75-3ec2-499e-9f30-6b8ae58255a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:37.545718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ae75-3ec2-499e-9f30-6b8ae58255a6 admin admin] PATH_INFO: `/v3/projects/dd9c395d11c54901859e6081ea0afcd9/users/789b98ddfef744239d765d3e2280aae0/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:37.548780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9ae75-3ec2-499e-9f30-6b8ae58255a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9c395d11c54901859e6081ea0afcd9, user_id=789b98ddfef744239d765d3e2280aae0, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:37.551460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9ae75-3ec2-499e-9f30-6b8ae58255a6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:37.561538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1223/2441] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 12:00:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:00:37.567709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1219/2442] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:00:37 2022] PUT /identity/v3/projects/dd9c395d11c54901859e6081ea0afcd9/users/789b98ddfef744239d765d3e2280aae0/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:00:37.594195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:37.595636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:37.613496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:37.742915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51a920-04fb-495e-9938-2c61d5cd1672 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:37.884256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:37.884934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:37.885103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:37.885429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:38.013966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:38.015772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:38.041462 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1224/2443] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:00:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:00:45.719545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d4ce8-8ed1-42a2-9572-cc8e8b96a27c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:45.720340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d4ce8-8ed1-42a2-9572-cc8e8b96a27c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:45.720340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d4ce8-8ed1-42a2-9572-cc8e8b96a27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:45.759535 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3f7d4ce8-8ed1-42a2-9572-cc8e8b96a27c None None] MFA Rules not processed for user `789b98ddfef744239d765d3e2280aae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:00:45.761099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3f7d4ce8-8ed1-42a2-9572-cc8e8b96a27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:45.853911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:45.902936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1220/2444] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:00:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:00:45.918475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:45.920060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:46.023683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=100 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:46.052143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bbda9-f332-4ebb-9d8c-120b39d14d06 service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:46.062622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:46.063288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:46.063574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:46.064198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:46.156761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:46.158977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:46.181039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1225/2445] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:55.876681 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:56.180764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=32 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:01.779786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f064-beaa-4e50-95e1-0f9494d4ec7d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:01.780275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f064-beaa-4e50-95e1-0f9494d4ec7d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:01.780369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f064-beaa-4e50-95e1-0f9494d4ec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:01.799571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1563f064-beaa-4e50-95e1-0f9494d4ec7d None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:01.800802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1563f064-beaa-4e50-95e1-0f9494d4ec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:01.809280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1221/2446] 10.222.0.88 () {58 vars in 967 bytes} [Mon Sep 19 12:01:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:01.821291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:01.832332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744195-5712-4e7a-8e4b-15b4c13407c9 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:01.838132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:01.838809 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:01.839108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:01.839672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:01.847995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:01.849546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:01.858081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1226/2447] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:01:04.885394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:04.885999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:04.885999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:04.904883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] MFA Rules not processed for user `fc84618366b045658728196980072790`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:04.906104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:04.912799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1222/2448] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:01:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:11.854353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:14.899654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:25.563104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:25.564502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:25.621078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7422e-5d8d-4a91-aebb-b0fd5be3fc3a service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:25.624574 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:25.625196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:25.625474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:25.625743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:25.627537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:25.641389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:25.642841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:25.653530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1227/2449] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:35.648875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=26 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:36.369166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:36.370679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:36.426757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f328442-8156-4dd4-97e5-b64b084fa94a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.430279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.430880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.431159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.431433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.433150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:36.492511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.493990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.503011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1223/2450] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:40.256930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:40.266683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031a50-87a2-4606-be8b-b2e16a822b6b service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:40.270389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:40.270975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:40.271249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:40.271547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:40.273282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:40.334579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:40.335986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:40.345243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1228/2451] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:45.558772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:45.571570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c3c34-3aeb-4e3a-8d54-178d5451ff1a service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:45.576979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:45.577569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:45.577879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:45.578151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:45.579981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:45.645235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:45.646845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:45.658067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1224/2452] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:50.340936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=28 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:55.651840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=76 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:56.298908 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:56.308350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb2782-a724-4c72-9be9-03953681536a service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:56.311916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:56.312505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:56.312789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:56.313087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:56.314952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:56.329152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:56.330593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:56.341664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1229/2453] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:06.336728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:35.654960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:35.664826 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c53b-2d82-4a59-8975-5af589e389c8 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:35.669160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ca689-3079-43ff-bbff-070154ef4b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:35.669783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ca689-3079-43ff-bbff-070154ef4b4b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:35.670072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ca689-3079-43ff-bbff-070154ef4b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:35.670348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ca689-3079-43ff-bbff-070154ef4b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:35.672083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-597ca689-3079-43ff-bbff-070154ef4b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:35.688170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ca689-3079-43ff-bbff-070154ef4b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:35.690868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ca689-3079-43ff-bbff-070154ef4b4b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:35.700025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1225/2454] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:45.696478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:55.020565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:55.022011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:55.033294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9358561-2d98-4f58-9d88-48e98b7e9270 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:55.036931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526718f-baed-418d-b57e-92f43edb3c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:55.037524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526718f-baed-418d-b57e-92f43edb3c27 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:55.037834 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526718f-baed-418d-b57e-92f43edb3c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:55.038105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526718f-baed-418d-b57e-92f43edb3c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:55.039911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9526718f-baed-418d-b57e-92f43edb3c27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:55.107792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526718f-baed-418d-b57e-92f43edb3c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:55.109205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526718f-baed-418d-b57e-92f43edb3c27 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:55.118880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1230/2455] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:04:05.114726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=28 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:33.912561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c801e7-628c-4c99-8789-7912247ce87a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:33.913156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c801e7-628c-4c99-8789-7912247ce87a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:33.913156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c801e7-628c-4c99-8789-7912247ce87a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:33.914467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1226/2456] 10.222.0.88 () {54 vars in 961 bytes} [Mon Sep 19 12:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 12:05:33.919336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7deafa-f1d0-4600-9776-07c49829bb50 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:33.919796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7deafa-f1d0-4600-9776-07c49829bb50 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:33.919915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7deafa-f1d0-4600-9776-07c49829bb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:33.999004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-1a7deafa-f1d0-4600-9776-07c49829bb50 None None] MFA Rules not processed for user `e38726440e2b444d9e2ccbb8681b9ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:34.000406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1a7deafa-f1d0-4600-9776-07c49829bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:34.112286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1231/2457] 10.222.0.88 () {58 vars in 1058 bytes} [Mon Sep 19 12:05:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:34.120434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c801e7-628c-4c99-8789-7912247ce87a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:34.136173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c801e7-628c-4c99-8789-7912247ce87a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:34.142203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:34.142809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:34.143086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:34.143352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:34.152438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:34.154028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:34.165068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1227/2458] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:44.110544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=74 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:44.160021 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:46.283448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7deafa-f1d0-4600-9776-07c49829bb50 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:46.315091 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7deafa-f1d0-4600-9776-07c49829bb50 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:46.320628 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:46.321333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:46.321674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:46.321918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:46.330889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:46.332438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:46.346221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1232/2459] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:56.337397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=13 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:25.000813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:25.021044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af0048-a07e-4031-8873-f9e373133fbb service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:25.024716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:25.025341 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:25.025599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:25.025905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:25.036420 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:25.037879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:25.049003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1228/2460] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:35.044107 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=10 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:35.761945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:35.762488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:35.789917 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674a02b-acb1-49e0-9466-61fab6ae8c66 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:35.796421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:35.797024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:35.797304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:35.797567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:35.806304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:35.807853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:35.836214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1233/2461] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:39.162707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:39.174063 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf638e1-4524-4688-bf5e-097281e9672e service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:39.180302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:39.180916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:39.181197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:39.181471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:39.190145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:39.191841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:39.201307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1229/2462] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:44.885591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:44.905260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6cc8b-8262-4784-ae34-f442cb63243d service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:44.908780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:44.909377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:44.909651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:44.910052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:44.919407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:44.920871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:44.948823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1234/2463] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:49.196819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:54.946200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=35 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:55.749288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:55.758948 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78c82b-6456-4b2a-870b-646e4be926fd service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:55.762700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:55.763300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:55.763576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:55.763843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:55.774218 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:55.775605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:55.787111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1230/2464] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:08:05.781520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:08:41.634687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:08:41.645397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880505-bf06-484f-83d6-5db3ebcf6ddd service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:08:41.649733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:08:41.650370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:08:41.650649 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:08:41.650914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:08:41.658865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:08:41.660258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:08:41.668977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1235/2465] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:08:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:08:51.664897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:08:55.385407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:08:55.395696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea7726-21f7-4120-a250-eaa5006f6bcb service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:08:55.399268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:08:55.399869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:08:55.400151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:08:55.400421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:08:55.409551 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:08:55.411600 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:08:55.439737 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1231/2466] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:08:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:09:05.436938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=13 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:10:39.963506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:10:39.965324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:10:39.979973 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e457c4-1412-4085-9411-81fb1749975e service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:10:39.984777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:10:39.985357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:10:39.985630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:10:39.985930 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:10:39.987626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:10:39.998804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:10:40.000329 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:10:40.009660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1236/2467] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:10:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:10:46.306768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:10:46.308340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:10:46.392916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858e79a-20e8-4721-9cec-9fe74316bddf service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:10:46.399061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:10:46.400251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:10:46.400553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:10:46.400828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:10:46.402739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:10:46.498751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:10:46.500414 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:10:46.518807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1232/2468] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:10:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:10:50.005674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:10:56.516050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=69 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:12:25.332409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:12:25.342024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f71f0-f2a0-439c-9943-f4e5edee20b8 service placement] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:12:25.345683 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:12:25.346391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:12:25.346729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:12:25.347060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:12:25.357423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:12:25.359381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:12:25.370411 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1237/2469] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:12:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:12:35.307409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:12:35.319597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fed0ba-6204-4b3c-969c-43700fbe2caa service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:12:35.325911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:12:35.326652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:12:35.327470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:12:35.327746 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:12:35.336296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:12:35.337887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:12:35.347057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1233/2470] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:12:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:12:35.365321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:12:40.047405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:12:40.058424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a183574-3db8-447b-9275-41c098b23cce service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:12:40.063802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:12:40.064392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:12:40.064668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:12:40.064940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:12:40.072737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:12:40.074349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:12:40.102806 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1238/2471] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:12:45.342846 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:12:46.176749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:12:46.178464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:12:46.265384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7abcb-c967-4b39-99fd-217e00285f1e service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:12:46.269138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:12:46.269743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:12:46.270059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:12:46.270328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:12:46.272159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:12:46.354424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:12:46.355867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:12:46.366350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1234/2472] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:12:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:12:50.099971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=13 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:12:56.301488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:12:56.303090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:12:56.362651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=58 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:12:56.373971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f80de-78e1-4f22-bff4-12119cddf46f service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:12:56.377660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:12:56.378290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:12:56.378598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:12:56.378872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:12:56.380746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:12:56.394611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:12:56.396293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:12:56.409430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1239/2473] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:12:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:13:06.403207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=31 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:13:11.453457 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:13:11.470644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba754dd2-aac3-448e-ac9f-2d0e5142fa22 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:13:11.477209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:13:11.478191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:13:11.479966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:13:11.480459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:13:11.482954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:13:11.496269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:13:11.498468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:13:11.526733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1235/2474] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:13:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:13:11.937954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:13:11.990117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f1efe-cd8f-43e6-8a46-ba2aeed32e84 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:13:12.004162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:13:12.005915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:13:12.006433 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:13:12.007899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:13:12.013255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:13:12.065689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:13:12.067590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:13:12.090187 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1240/2475] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:13:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:13:21.506962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:13:22.075701 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:13:46.484747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:13:46.487006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:13:46.498168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eaa0c-f365-4577-b68e-fc175c25dd68 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:13:46.501715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:13:46.502346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:13:46.502622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:13:46.502889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:13:46.504662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:13:46.514482 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:13:46.516067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:13:46.524796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1236/2476] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:13:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:13:56.268532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:13:56.279853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e0a9-7720-413b-a428-58cc73361794 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:13:56.286040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:13:56.286635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:13:56.286981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:13:56.287260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:13:56.289105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:13:56.372921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:13:56.374537 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:13:56.385446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1241/2477] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:13:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:13:56.521120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:14:06.380223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=33 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:15:44.883290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:15:45.004823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86c0cf-7857-4b09-90f4-d19772f40cdb service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:15:45.010800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:15:45.011627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:15:45.011917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:15:45.012193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:15:45.026511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:15:45.026511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:15:45.034452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1237/2478] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:15:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:15:45.077048 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:15:45.079487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:15:45.113784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d42fb-03f3-4b5f-a845-96627955c668 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:15:45.117501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:15:45.118138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:15:45.118420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:15:45.118695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:15:45.128272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:15:45.130620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:15:45.140694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1242/2479] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:15:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:15:55.030550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=54 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:15:55.136366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=15 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:17:25.372917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:17:25.391751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f21-aedf-4343-9636-ef88f4ac6728 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:17:25.395816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:17:25.396411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:17:25.396679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:17:25.396946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:17:25.398768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:17:25.420595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:17:25.422045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:17:25.443711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1238/2480] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:17:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:17:35.410342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:17:35.421147 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1bd7-5a63-4131-831d-af451961b15c service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:17:35.425694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:17:35.426324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:17:35.426601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:17:35.426872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:17:35.428728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:17:35.440699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=17 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:17:35.505594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:17:35.507063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:17:35.517443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1243/2481] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:17:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:17:41.079307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:17:41.091026 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc040b-bfd3-44c0-86f8-9bb4c1700de1 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:17:41.094683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:17:41.095300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:17:41.095579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:17:41.095853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:17:41.097927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:17:41.181844 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:17:41.183739 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:17:41.193447 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1239/2482] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:17:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:17:45.512231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=33 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:17:47.082637 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:17:47.095253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39aee8f-30c9-4da8-8a09-103f07317620 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:17:47.100661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:17:47.101256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:17:47.101530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:17:47.101822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:17:47.110354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:17:47.112392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:17:47.140101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1244/2483] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:17:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:17:51.189060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=33 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:17:56.659113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:17:56.669382 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ad0f5-13b4-4e1d-b8da-bcd24607ca2a service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:17:56.673082 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:17:56.673740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:17:56.674051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:17:56.674327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:17:56.688701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:17:56.690224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:17:56.702226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1240/2484] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:17:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:17:57.137440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=13 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:18:06.696999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:18:53.629140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:18:53.644905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5d1ad-27fc-480e-bdb9-f12fd64131a0 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:18:53.650732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:18:53.651385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:18:53.651673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:18:53.652064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:18:53.666903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:18:53.668999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:18:53.680273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1245/2485] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:18:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:18:57.079972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:18:57.081675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:18:57.110654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b501fb-29db-4e91-aff9-d974b66fe388 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:18:57.116088 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:18:57.116686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:18:57.116962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:18:57.117229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:18:57.125214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:18:57.126901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:18:57.155617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1241/2486] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:18:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:03.674152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:19:05.634538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:05.643534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc03a-c55e-4c58-a3bf-6893ce37047b service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:05.647230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:05.647886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:05.648107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:05.648374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:05.650229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:05.736196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:05.737603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:05.756421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1246/2487] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:07.152259 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=18 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:19:09.363967 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:09.375011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6caec3d-e1ad-4821-b192-ff73c8d2766b service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:09.379869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:09.380463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:09.380741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:09.381122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:09.382868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:09.413275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:09.415263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:09.437875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1242/2488] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:12.299703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:12.328566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc675f7-88fa-4a01-b628-d8ae74714aec service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:12.336801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:12.339723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:12.340014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:12.340292 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:12.365996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:12.368261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:12.385052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1247/2489] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:13.785802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:13.798566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8da827-ba9c-43bc-bade-0970b9a72327 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:13.806546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:13.807637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:13.808146 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:13.808508 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:13.810940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:13.825703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:13.827557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:13.843723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1243/2490] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:13.854318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:13.856357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:13.877023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb57fee-2e60-43f9-9590-72e3766baf04 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:13.881571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:13.882367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:13.882722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:13.883066 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] PATH_INFO: `/v3/users/fc84618366b045658728196980072790` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:13.884358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc84618366b045658728196980072790)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:13.886368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:13.937902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc84618366b045658728196980072790'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:13.947195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc84618366b045658728196980072790'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:13.948983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc84618366b045658728196980072790'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:13.956496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc84618366b045658728196980072790'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:13.970904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc84618366b045658728196980072790'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:13.978885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1248/2491] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:19:13 2022] DELETE /identity/v3/users/fc84618366b045658728196980072790 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:19:14.221696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:14.242655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54bea-7456-448b-989a-1b70e81ba08b service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:14.314988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:14.315633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:14.315917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:14.316191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] PATH_INFO: `/v3/projects/2f7dd7174903460989484c1ab233208a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:14.317326 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7dd7174903460989484c1ab233208a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:14.319405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:14.370837 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7dd7174903460989484c1ab233208a', 'request_id': 'req-75932feb-ff8c-4e39-9687-bb3b985b0399'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.377811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7dd7174903460989484c1ab233208a', 'request_id': 'req-75932feb-ff8c-4e39-9687-bb3b985b0399'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.383614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7dd7174903460989484c1ab233208a', 'request_id': 'req-75932feb-ff8c-4e39-9687-bb3b985b0399'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.398913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] The token cache is being invalidate because project 2f7dd7174903460989484c1ab233208a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:19:14.399209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.400097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.402279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1244/2492] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:19:14 2022] DELETE /identity/v3/projects/2f7dd7174903460989484c1ab233208a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:19:14.406496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:14.407953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:14.421054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02f604-b2db-4c65-ac0f-6a4790ad8bfd admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:14.441385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:19:14.492907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:14.493879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:14.494348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:14.494627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] PATH_INFO: `/v3/users/789b98ddfef744239d765d3e2280aae0` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:14.510892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=789b98ddfef744239d765d3e2280aae0)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:14.513410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:14.563679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789b98ddfef744239d765d3e2280aae0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.571099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789b98ddfef744239d765d3e2280aae0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.571706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789b98ddfef744239d765d3e2280aae0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.580197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789b98ddfef744239d765d3e2280aae0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.592775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789b98ddfef744239d765d3e2280aae0'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.598886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1249/2493] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:19:14 2022] DELETE /identity/v3/users/789b98ddfef744239d765d3e2280aae0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:19:14.783338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:14.784794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:14.807710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932feb-ff8c-4e39-9687-bb3b985b0399 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:14.868385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:19:14.879817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:14.880498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:14.880786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:14.881056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] PATH_INFO: `/v3/projects/dd9c395d11c54901859e6081ea0afcd9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:14.892106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9c395d11c54901859e6081ea0afcd9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:14.894040 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:14.944343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9c395d11c54901859e6081ea0afcd9', 'request_id': 'req-9ff51c13-ba60-434b-8f75-c2e62fd63a70'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.952097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9c395d11c54901859e6081ea0afcd9', 'request_id': 'req-9ff51c13-ba60-434b-8f75-c2e62fd63a70'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.957733 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9c395d11c54901859e6081ea0afcd9', 'request_id': 'req-9ff51c13-ba60-434b-8f75-c2e62fd63a70'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.971756 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] The token cache is being invalidate because project dd9c395d11c54901859e6081ea0afcd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:19:14.972054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.972711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ff51c13-ba60-434b-8f75-c2e62fd63a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:19:14.974477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1245/2494] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:19:14 2022] DELETE /identity/v3/projects/dd9c395d11c54901859e6081ea0afcd9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:19:24.595256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=47,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:19:24.970307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:19:43.995452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:43.999379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:44.109470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0979-69a8-4507-bca5-11eb1cd63006 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:44.207927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:44.208502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:44.208790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:44.209095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:44.210949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:44.233160 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:44.235999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:44.261832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1250/2495] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:45.956060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:45.957085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:45.957085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:45.987436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] MFA Rules not processed for user `766511b86ad643c099998fc4e1638f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:19:45.988659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:46.050729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1246/2496] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:19:46.402291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:46.404279 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:46.471369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f623643-8b5d-4b30-a018-a9f250e2df92 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:46.476013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:46.476865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:46.477366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:46.477754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:46.480069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:46.491511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:46.492928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:46.506491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1251/2497] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:46.869489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:46.872305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:46.965340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7062-c278-4f31-93a8-f273b893cc9b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:46.970400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:46.971036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:46.971318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:46.971710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:46.973398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:46.985498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:46.987147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:46.998392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1247/2498] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:19:55.115938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:19:55.117515 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:55.146136 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c850a-4991-46d7-9aa8-68e5c9a164ab service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:19:55.155082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:19:55.155807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:19:55.156063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:19:55.156315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:19:55.158294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:19:55.184693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:19:55.188236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:19:55.200595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1252/2499] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:19:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:19:56.993417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=60 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:20:05.194954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=94 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:20:48.880553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:20:48.882281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:20:48.894838 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed350c9-5f9d-48d2-9fe3-178baa5476e0 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:20:48.899831 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:20:48.900423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:20:48.901683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:20:48.901936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:20:48.904918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:20:48.919713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:20:48.921258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:20:48.930888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1248/2500] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:20:58.926848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:22:26.714409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:22:26.723563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a6e9-7a08-4093-aed1-c201e1a08ad3 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:22:26.727078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:22:26.727659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:22:26.727933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:22:26.728200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:22:26.729910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:22:26.742594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:22:26.743976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:22:26.754453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1253/2501] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:22:35.565413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:22:35.566873 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:22:35.582243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e81d3f-cf65-47e5-9e21-d1f7f2e5149c service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:22:35.587892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:22:35.588487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:22:35.588770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:22:35.589079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:22:35.590774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:22:35.657205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:22:35.658817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:22:35.678527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1249/2502] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:22:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:22:36.749736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:22:45.675427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=36 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:22:56.766365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:22:56.775735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c89bc-69c3-488b-a2ee-bc0df360fac1 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:22:56.779484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:22:56.780100 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:22:56.780374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:22:56.780643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:22:56.782337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:22:56.797030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:22:56.798476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:22:56.809359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1254/2503] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:23:06.804377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:23:12.217757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:23:12.219148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:23:12.300821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44fe54-4545-41db-a0a1-0db9b3f17b4b service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:23:12.305148 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:23:12.305785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:23:12.306081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:23:12.306353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:23:12.308286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:23:12.386288 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:23:12.387740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:23:12.408317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1250/2504] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:23:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:23:12.502956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:23:12.504489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:23:12.517635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6eb6e-1c0a-49ba-84f9-9d685d7f8a67 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:23:12.522678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:23:12.523787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:23:12.523907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:23:12.524235 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:23:12.526412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:23:12.555309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:23:12.556727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:23:12.566320 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1255/2505] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:23:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:23:22.405101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=65 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:23:22.561920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=11 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:24:01.616303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:24:01.617927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:24:01.641208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7e562-3137-4438-b526-78c5a674bf23 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:24:01.645222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:24:01.645848 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:24:01.646132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:24:01.646406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:24:01.648111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:24:01.657731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:24:01.659825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:24:01.667750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1251/2506] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:24:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:24:11.664200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=11 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:25:12.657587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:12.670203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ab06b-9928-4334-910f-1a66fb734ff2 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:12.675396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:12.676090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:12.676380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:12.676660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:12.678671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:12.771499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:12.773072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:12.791095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1256/2507] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:12.791475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:12.802700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12aa601-5571-456b-9332-28e3dcc7f187 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:12.806409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dbb09-0376-424c-bf2e-70b0811c4d08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:12.807212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dbb09-0376-424c-bf2e-70b0811c4d08 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:12.807651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dbb09-0376-424c-bf2e-70b0811c4d08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:12.808180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dbb09-0376-424c-bf2e-70b0811c4d08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:12.819153 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dbb09-0376-424c-bf2e-70b0811c4d08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:12.820693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dbb09-0376-424c-bf2e-70b0811c4d08 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:12.842110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1252/2508] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:14.942327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:14.959219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc618a8-8a23-486b-ac05-0061b95044c2 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:14.969288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:14.969960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:14.970284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:14.970694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:14.972878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:14.988781 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:14.990575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:15.002978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1257/2509] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:15.340328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53033-1634-4d9d-ad9e-0860a98e6bf4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:15.340897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53033-1634-4d9d-ad9e-0860a98e6bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:15.340897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53033-1634-4d9d-ad9e-0860a98e6bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:15.374730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6ff53033-1634-4d9d-ad9e-0860a98e6bf4 None None] MFA Rules not processed for user `8fa4b6af47384c7fb26aae3547d02b64`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:25:15.377270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6ff53033-1634-4d9d-ad9e-0860a98e6bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:15.385211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1253/2510] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:25:15.725745 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:15.735048 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cdc4f-556f-44fb-9e4f-854382a85ad3 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:15.738555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:15.739139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:15.739418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:15.739688 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:15.741347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:15.750960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:15.752348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:15.760825 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1258/2511] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:16.092485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff53033-1634-4d9d-ad9e-0860a98e6bf4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:16.104332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff53033-1634-4d9d-ad9e-0860a98e6bf4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:16.109328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:16.109960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:16.110242 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:16.110514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:16.120807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:16.122542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:16.133427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1254/2512] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:19.015718 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:19.024485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a11654-6e49-4e46-bb25-48baba9d0c34 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:19.028182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:19.028800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:19.029080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:19.029357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:19.031115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:19.095801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:19.097202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:19.107030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1259/2513] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:22.519644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:22.534855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757cf7e-f793-4ce9-8ff4-fcaa6b4cc7d2 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:22.544138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:22.545269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:22.545742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:22.546309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:22.548387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:22.562461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:22.576834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:22.577208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:22.577208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:22.585320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1255/2514] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:22.592964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc579da-0855-4f82-bd98-e73b09d80b54 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:22.598238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:22.598949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:22.599304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:22.599651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] PATH_INFO: `/v3/users/11af9e7e3fed4bea9187dee0ebf27551` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:22.600850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11af9e7e3fed4bea9187dee0ebf27551)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:22.604077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:22.667966 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11af9e7e3fed4bea9187dee0ebf27551'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:22.671268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:25:22.679186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11af9e7e3fed4bea9187dee0ebf27551'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:22.680542 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11af9e7e3fed4bea9187dee0ebf27551'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:22.690763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11af9e7e3fed4bea9187dee0ebf27551'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:22.704421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-97317fd3-067b-458e-b4d1-d2b7fed5a256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11af9e7e3fed4bea9187dee0ebf27551'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:22.710392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1260/2515] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:25:22 2022] DELETE /identity/v3/users/11af9e7e3fed4bea9187dee0ebf27551 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:25:22.890851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:22.902795 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7673d2-489e-492b-9811-a388df2e8e1c service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:22.963035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:22.963905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:22.964190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:22.964461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] PATH_INFO: `/v3/projects/bf7ef0ac87cb450f9320b8d3a313d942` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:22.965589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7ef0ac87cb450f9320b8d3a313d942)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:22.967515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.014041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7ef0ac87cb450f9320b8d3a313d942', 'request_id': 'req-a0f7a649-29b4-4910-8c37-a7bb44b6716f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:23.020602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7ef0ac87cb450f9320b8d3a313d942', 'request_id': 'req-a0f7a649-29b4-4910-8c37-a7bb44b6716f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:23.026499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7ef0ac87cb450f9320b8d3a313d942', 'request_id': 'req-a0f7a649-29b4-4910-8c37-a7bb44b6716f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:23.042086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] The token cache is being invalidate because project bf7ef0ac87cb450f9320b8d3a313d942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:25:23.042517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:23.043184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0f7a649-29b4-4910-8c37-a7bb44b6716f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:25:23.045329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1256/2516] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:25:22 2022] DELETE /identity/v3/projects/bf7ef0ac87cb450f9320b8d3a313d942 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:25:23.051078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2ae03-9fc7-4a56-8ada-ac1916f9e5c3 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.051444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2ae03-9fc7-4a56-8ada-ac1916f9e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.051638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2ae03-9fc7-4a56-8ada-ac1916f9e5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.115286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ccd2ae03-9fc7-4a56-8ada-ac1916f9e5c3 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:25:23.116533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ccd2ae03-9fc7-4a56-8ada-ac1916f9e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:23.189335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1261/2517] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:25:23.203054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefbdec-f340-4c0d-8925-0b6c2878da1b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.203516 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefbdec-f340-4c0d-8925-0b6c2878da1b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.203800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefbdec-f340-4c0d-8925-0b6c2878da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.224759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6cefbdec-f340-4c0d-8925-0b6c2878da1b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:25:23.225608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6cefbdec-f340-4c0d-8925-0b6c2878da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:23.235263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1257/2518] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:25:23.239640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2ae03-9fc7-4a56-8ada-ac1916f9e5c3 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:23.249992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2ae03-9fc7-4a56-8ada-ac1916f9e5c3 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:23.254559 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:23.255164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.255439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.255707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.256350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:25:23.256657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:23.258326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.269977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1262/2519] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:25:23.274239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefbdec-f340-4c0d-8925-0b6c2878da1b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:23.283340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefbdec-f340-4c0d-8925-0b6c2878da1b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:23.287981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:23.309738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.309738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.309738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.309738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:23.309738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.332131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:25:23.340410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1258/2520] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:25:23 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:25:23.344709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:23.355003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70a9dd-3a6a-4dfd-9199-ef05136e4185 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:23.409668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:23.410290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.410569 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.410841 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.411458 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:23.413338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.436600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1263/2521] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:25:23 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:25:23.442156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:23.453701 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e96de-dbae-4865-befd-1f56ac786413 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:23.458299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:23.458926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.459210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.459486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.460126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:23.461708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.472296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1259/2522] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:25:23.476288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:23.487246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221e301-a2a6-4a59-a855-6166566d2a43 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:23.492015 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bd5b0-d604-4d49-8634-cae931820ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:23.492654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bd5b0-d604-4d49-8634-cae931820ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.492928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bd5b0-d604-4d49-8634-cae931820ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.493798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bd5b0-d604-4d49-8634-cae931820ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.494988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bd5b0-d604-4d49-8634-cae931820ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:23.496181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bd5b0-d604-4d49-8634-cae931820ed3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.506671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1264/2523] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:25:23.510903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:23.520779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a59b1-98b5-4e31-b448-78dda5ac78a2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:23.526192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:23.526841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.527122 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.527402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] PATH_INFO: `/v3/projects/c5ea64acca284efda929aca51f9280f4/users/3e949d775a6a4b4b87f665a863e6ddf3/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.543685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ea64acca284efda929aca51f9280f4, user_id=3e949d775a6a4b4b87f665a863e6ddf3, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:23.546399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.559229 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1260/2524] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:25:23 2022] PUT /identity/v3/projects/c5ea64acca284efda929aca51f9280f4/users/3e949d775a6a4b4b87f665a863e6ddf3/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:25:23.570354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.570640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.570939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.578326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:25:23.604078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] MFA Rules not processed for user `3e949d775a6a4b4b87f665a863e6ddf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:25:23.605284 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:23.669409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1265/2525] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:25:23.680433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:23.682316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:23.755134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabf340-1365-4ae9-97a6-af0e1ffa3c53 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:23.760679 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:23.761281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:23.761565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:23.761863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:23.769421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:23.771020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:23.780442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1261/2526] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:24.679712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:24.682009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:24.700378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19714da-8bf3-4643-83a7-209f7936ef06 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:24.705912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:24.706514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:24.706794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:24.707070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:24.708790 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:24.721218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:24.722818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:24.735979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1266/2527] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:28.536813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:28.549053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cdb82-6872-4f20-8b94-52bc283a9cf2 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:28.554799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:28.555517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:28.556265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:28.556629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:28.558682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:28.571394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:28.572982 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:28.584274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1262/2528] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:25:34.729324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=37 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:25:38.579462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:25:54.872319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:25:54.873744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:54.997594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b416d-594f-4091-b749-b23c14e8b832 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:25:55.002707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1c914-e850-4709-b0b0-e29ab24d9341 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:25:55.003308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1c914-e850-4709-b0b0-e29ab24d9341 service placement] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:25:55.003587 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1c914-e850-4709-b0b0-e29ab24d9341 service placement] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:25:55.003859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1c914-e850-4709-b0b0-e29ab24d9341 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:25:55.005560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d6a1c914-e850-4709-b0b0-e29ab24d9341 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:25:55.017026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1c914-e850-4709-b0b0-e29ab24d9341 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:25:55.018627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1c914-e850-4709-b0b0-e29ab24d9341 service placement] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:25:55.029191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1267/2529] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:25:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:26:05.024405 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=58 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:26:23.744655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:23.831767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70664be6-8b42-4648-af2e-03ba6a3ff364 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:23.836410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:23.837294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:23.837584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:23.837890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] PATH_INFO: `/v3/users/3e949d775a6a4b4b87f665a863e6ddf3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:23.839232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e949d775a6a4b4b87f665a863e6ddf3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:23.841008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:23.892928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e949d775a6a4b4b87f665a863e6ddf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:23.893583 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e949d775a6a4b4b87f665a863e6ddf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:23.900936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e949d775a6a4b4b87f665a863e6ddf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:23.909355 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e949d775a6a4b4b87f665a863e6ddf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:23.922002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e949d775a6a4b4b87f665a863e6ddf3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:23.927767 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1263/2530] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:26:23 2022] DELETE /identity/v3/users/3e949d775a6a4b4b87f665a863e6ddf3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:26:23.933314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:23.933948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:23.933948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:23.956877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:26:23.958139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:24.032503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1268/2531] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:26:24.042279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.043686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:24.102944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b4187-5c4e-4539-9ac3-f4ccef3f0a8d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.106937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e9bb2-cb16-496a-90ea-30e694be32a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.107575 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e9bb2-cb16-496a-90ea-30e694be32a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.107853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e9bb2-cb16-496a-90ea-30e694be32a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.108125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e9bb2-cb16-496a-90ea-30e694be32a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.117116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e9bb2-cb16-496a-90ea-30e694be32a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.118621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e9bb2-cb16-496a-90ea-30e694be32a8 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.123406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:26:24.131517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1264/2532] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:26:24.411601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.413016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:24.425132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d7a4-3ac3-4047-a67e-42c5374d439f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.429886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.430546 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.430828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.431103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] PATH_INFO: `/v3/projects/c5ea64acca284efda929aca51f9280f4` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.432862 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ea64acca284efda929aca51f9280f4)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.434802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.484385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ea64acca284efda929aca51f9280f4', 'request_id': 'req-7be76da4-3789-4d08-963b-b700823bc114'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:24.491848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ea64acca284efda929aca51f9280f4', 'request_id': 'req-7be76da4-3789-4d08-963b-b700823bc114'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:24.498702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ea64acca284efda929aca51f9280f4', 'request_id': 'req-7be76da4-3789-4d08-963b-b700823bc114'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:24.508873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] The token cache is being invalidate because project c5ea64acca284efda929aca51f9280f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:26:24.509161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:24.509832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-7be76da4-3789-4d08-963b-b700823bc114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:26:24.511302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1269/2533] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:26:24 2022] DELETE /identity/v3/projects/c5ea64acca284efda929aca51f9280f4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:26:24.516169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e939bf4-c4a1-4be4-aca9-8a00c4da4437 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.516453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e939bf4-c4a1-4be4-aca9-8a00c4da4437 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.516723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e939bf4-c4a1-4be4-aca9-8a00c4da4437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.536869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1e939bf4-c4a1-4be4-aca9-8a00c4da4437 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:26:24.538139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1e939bf4-c4a1-4be4-aca9-8a00c4da4437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:24.596944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1265/2534] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:26:24.605389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed645-094d-4df0-86c9-90e4bc2e2488 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.605673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed645-094d-4df0-86c9-90e4bc2e2488 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.605971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed645-094d-4df0-86c9-90e4bc2e2488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.625059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-f98ed645-094d-4df0-86c9-90e4bc2e2488 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:26:24.626268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f98ed645-094d-4df0-86c9-90e4bc2e2488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:24.633883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1270/2535] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:26:24.638015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e939bf4-c4a1-4be4-aca9-8a00c4da4437 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.647491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e939bf4-c4a1-4be4-aca9-8a00c4da4437 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.652647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.653238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.653519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.653823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.654471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:26:24.654779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.656375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.668332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1266/2536] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:26:24.672738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ed645-094d-4df0-86c9-90e4bc2e2488 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.682052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ed645-094d-4df0-86c9-90e4bc2e2488 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.686735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.687354 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.687639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.687911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.688550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.690463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.717464 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1271/2537] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:26:24.721518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.731930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ae76-df74-46c3-a254-ac858745372d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.787376 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.788056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.788176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.788449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.789111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.791069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.813335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1267/2538] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:26:24 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:26:24.817367 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.827629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e76c-d2c7-4025-ab92-8a4c2503a120 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.832334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.832932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.833215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.833488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.834168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.835737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.846304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1272/2539] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:26:24.850198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.860346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9fad-627a-45a2-9260-42ec7a7b4b38 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.864839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ef6b-4f03-4af1-adb5-fe610aa9a233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.865421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ef6b-4f03-4af1-adb5-fe610aa9a233 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.865697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ef6b-4f03-4af1-adb5-fe610aa9a233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.865995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ef6b-4f03-4af1-adb5-fe610aa9a233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.866623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ef6b-4f03-4af1-adb5-fe610aa9a233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.868188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ef6b-4f03-4af1-adb5-fe610aa9a233 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.879377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1268/2540] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:26:24.883255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:24.892764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397174a6-6e63-48df-8ed0-d1396d699b9e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:24.897430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:24.898121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.898392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.898661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] PATH_INFO: `/v3/projects/c6a31eac559e4ada9bdf241b7dd95aab/users/404304f411544ae79e68720c90c3adc3/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.913326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a31eac559e4ada9bdf241b7dd95aab, user_id=404304f411544ae79e68720c90c3adc3, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:24.915957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:24.928850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1273/2541] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:26:24 2022] PUT /identity/v3/projects/c6a31eac559e4ada9bdf241b7dd95aab/users/404304f411544ae79e68720c90c3adc3/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:26:24.940201 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd24b7b-51f8-4066-a297-5ceef84ea5c4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:24.940488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd24b7b-51f8-4066-a297-5ceef84ea5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:24.940762 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd24b7b-51f8-4066-a297-5ceef84ea5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:24.969582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5dd24b7b-51f8-4066-a297-5ceef84ea5c4 None None] MFA Rules not processed for user `404304f411544ae79e68720c90c3adc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:26:24.970807 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5dd24b7b-51f8-4066-a297-5ceef84ea5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:25.024174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:26:25.038477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1269/2542] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:26:25.043490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.064213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:26:25.106931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22466a-bb34-4930-830c-212f4d08252d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.111737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.112366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.112648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.112926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.113576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.115548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.142555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1274/2543] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:26:25 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:26:25.146669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd24b7b-51f8-4066-a297-5ceef84ea5c4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.158412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd24b7b-51f8-4066-a297-5ceef84ea5c4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.215025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.215632 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.215910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.216180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.216818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.218782 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.238923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1270/2544] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:26:25 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:26:25.242982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.253122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe5f89-3311-47c7-8bd9-1236370600c3 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.257796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.258389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.258666 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.258937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.259575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.261687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.271962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1275/2545] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:26:25.276480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.286190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246d1b-5096-4de7-ac9c-6d6aa751b7f7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.290813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4e92a-524e-40bc-aad7-8e892d070c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.291473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4e92a-524e-40bc-aad7-8e892d070c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.291734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4e92a-524e-40bc-aad7-8e892d070c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.292007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4e92a-524e-40bc-aad7-8e892d070c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.292651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4e92a-524e-40bc-aad7-8e892d070c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.294266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4e92a-524e-40bc-aad7-8e892d070c03 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.304099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1271/2546] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:26:25.308877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.318762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1885de-0547-499c-8066-9e9081012532 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.324112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.324753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.325030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.325300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] PATH_INFO: `/v3/projects/5a41ece4975047868c03a143201aa4a7/users/e9fe8511ce144954b4156f7174b70c0b/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.339968 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a41ece4975047868c03a143201aa4a7, user_id=e9fe8511ce144954b4156f7174b70c0b, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.342615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.355797 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1276/2547] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:26:25 2022] PUT /identity/v3/projects/5a41ece4975047868c03a143201aa4a7/users/e9fe8511ce144954b4156f7174b70c0b/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:26:25.367047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5257d03-d539-4521-9acf-1eec15bba974 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.367361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5257d03-d539-4521-9acf-1eec15bba974 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.367633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5257d03-d539-4521-9acf-1eec15bba974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.396659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a5257d03-d539-4521-9acf-1eec15bba974 None None] MFA Rules not processed for user `e9fe8511ce144954b4156f7174b70c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:26:25.397977 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a5257d03-d539-4521-9acf-1eec15bba974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:25.459804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1272/2548] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:26:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:26:25.464530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.524610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59703557-8092-4691-9400-787a25e99e69 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.529268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.529896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.530199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.530467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.531082 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.532936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.559319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1277/2549] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:26:25 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:26:25.563424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5257d03-d539-4521-9acf-1eec15bba974 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.574299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5257d03-d539-4521-9acf-1eec15bba974 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.629145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.629748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.630060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.630336 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.630974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.632891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.642178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:26:25.655067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1273/2550] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:26:25 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:26:25.659039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.669179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57cab1-6594-48fb-907a-67d4607592fe admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.673736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.674362 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.674645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.674924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.675566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.677164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.688172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1278/2551] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:26:25.692384 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.702717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19d130-fdb4-4964-8aca-94cbe56a872e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.707369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.708039 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.708321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.708597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] PATH_INFO: `/v3/projects/79b278a5a4364895a59b5e06bf89c4dd/users/bbe7d3ceb4fc448c93d72ce46c73ec81/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.723210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b278a5a4364895a59b5e06bf89c4dd, user_id=bbe7d3ceb4fc448c93d72ce46c73ec81, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.725887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.738860 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1274/2552] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:26:25 2022] PUT /identity/v3/projects/79b278a5a4364895a59b5e06bf89c4dd/users/bbe7d3ceb4fc448c93d72ce46c73ec81/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:26:25.742786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.753336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81944491-70fd-46d3-8502-992e3a68e342 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.792814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:26:25.808583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aff67-54e8-49c0-88ed-a88b36be77b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.809180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aff67-54e8-49c0-88ed-a88b36be77b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.809501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aff67-54e8-49c0-88ed-a88b36be77b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.809721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aff67-54e8-49c0-88ed-a88b36be77b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.810382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aff67-54e8-49c0-88ed-a88b36be77b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.811938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aff67-54e8-49c0-88ed-a88b36be77b6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.821334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1279/2553] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:26:25.825479 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:26:25.835684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43c30f-3c3d-47c0-bff9-b60551fa1259 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:26:25.840895 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:26:25.841546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.841857 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.842130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] PATH_INFO: `/v3/projects/79b278a5a4364895a59b5e06bf89c4dd/users/bbe7d3ceb4fc448c93d72ce46c73ec81/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.844018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b278a5a4364895a59b5e06bf89c4dd, user_id=bbe7d3ceb4fc448c93d72ce46c73ec81, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:26:25.846765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:26:25.857518 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1275/2554] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:26:25 2022] PUT /identity/v3/projects/79b278a5a4364895a59b5e06bf89c4dd/users/bbe7d3ceb4fc448c93d72ce46c73ec81/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:26:25.867424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:26:25.867711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:26:25.867986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:26:25.895948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] MFA Rules not processed for user `bbe7d3ceb4fc448c93d72ce46c73ec81`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:26:25.897195 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:26:25.963899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1280/2555] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:26:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:26:35.853571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=4,keystone:SELECT=17 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:26:35.959843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=43 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:27:14.261645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:27:14.263412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:27:14.320790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51480834-598f-43bb-ad28-17eb29f43239 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:27:14.325050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:27:14.325649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:27:14.325957 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:27:14.326234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:27:14.335126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:27:14.336541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:27:14.346260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1276/2556] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:27:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:27:24.342059 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=26 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:29:08.590887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:29:08.592340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:29:08.656894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87976df-d663-44d5-8da6-647e0848d1fa None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:29:08.660422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:29:08.661013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:29:08.661290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:29:08.661556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:29:08.663286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:29:08.672760 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:29:08.674234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:29:08.682050 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1281/2557] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:29:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:29:14.448643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:29:14.458695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939a87-66ef-4b5f-a36a-a9183754a529 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:29:14.462653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:29:14.463247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:29:14.463527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:29:14.463800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:29:14.465509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:29:14.567436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:29:14.568869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:29:14.591041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1277/2558] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:29:14.637912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:29:14.647928 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637004b-e195-4d77-b315-dc5c8c9955ff service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:29:14.651621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:29:14.652373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:29:14.652654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:29:14.652925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:29:14.654795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:29:14.724115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:29:14.725532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:29:14.744716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1282/2559] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:29:14.824402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:29:14.833716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ac4bd-9d9e-46e5-89a6-ab879ff5d918 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:29:14.837673 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:29:14.838301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:29:14.838584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:29:14.838858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:29:14.895330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:29:14.896802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:29:14.906156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1278/2560] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:29:14.935176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:29:14.945406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f33e94-effd-41aa-af36-f1ec9923042e service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:29:14.949929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:29:14.950647 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:29:14.950933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:29:14.951206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:29:14.953064 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:29:15.031451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:29:15.032857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:29:15.042481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1283/2561] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:29:24.901783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=74 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:29:25.037905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:30:56.824523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:30:56.826162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:30:56.910448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69127c-65b3-4a5a-ba41-bdbd4a031eb5 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:30:56.915962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:30:56.916685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:30:56.916868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:30:56.917143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:30:56.918940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:30:56.931545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:30:56.933127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:30:56.943778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1279/2562] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:31:06.938891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=37 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:31:13.291912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:13.293269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:13.304478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd25eb-a996-45d8-88c2-66cb4b360cff service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:13.307905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:13.308488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:13.308761 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:13.309026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:13.310737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:13.376645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:13.378606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:13.396056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1284/2563] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:31:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:31:15.024211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:15.025942 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:15.037783 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:15.039198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:15.060699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0111ba-57e5-4011-afbc-3bae13725fd4 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:15.073494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:15.075768 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:15.076060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:15.076582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bff67-a44b-4cdf-9186-88f8226119fb service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:15.078213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:15.082133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:15.084488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a835a0-335c-4b2e-be9c-6f82727b7ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:15.085212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a835a0-335c-4b2e-be9c-6f82727b7ae8 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:15.085530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a835a0-335c-4b2e-be9c-6f82727b7ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:15.085832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a835a0-335c-4b2e-be9c-6f82727b7ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:15.089079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-20a835a0-335c-4b2e-be9c-6f82727b7ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:15.104269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:15.107199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:15.115250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a835a0-335c-4b2e-be9c-6f82727b7ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:15.122347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1285/2564] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:31:15.126034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a835a0-335c-4b2e-be9c-6f82727b7ae8 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:15.164853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1280/2565] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:31:15.287269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:15.412934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6addfa-876d-4757-9499-38fbf4a0bb85 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:15.420239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:15.421056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:15.421409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:15.421597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] PATH_INFO: `/v3/users/404304f411544ae79e68720c90c3adc3` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:15.422778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404304f411544ae79e68720c90c3adc3)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:15.424857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:15.474558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404304f411544ae79e68720c90c3adc3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:15.478819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:31:15.483631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404304f411544ae79e68720c90c3adc3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:15.485272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404304f411544ae79e68720c90c3adc3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:15.496521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404304f411544ae79e68720c90c3adc3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:15.510595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404304f411544ae79e68720c90c3adc3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:15.516782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1286/2566] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:31:15 2022] DELETE /identity/v3/users/404304f411544ae79e68720c90c3adc3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:31:15.521506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:15.521816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:15.522096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:15.553075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:31:15.554327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:15.628804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1281/2567] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:31:15.635406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:15.636845 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:15.699754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be07b9c-75e7-4481-9f1d-dd4b555bd1b2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:15.703566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:15.704176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:15.704445 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:15.704712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:15.716285 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:15.718037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:15.730155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1287/2568] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:31:16.020192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:16.022522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:16.039412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929167c-4b19-42d8-bcb6-1895b98c3be5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:16.044363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:16.045048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:16.045358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:16.045635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] PATH_INFO: `/v3/projects/c6a31eac559e4ada9bdf241b7dd95aab` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:16.046827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a31eac559e4ada9bdf241b7dd95aab)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:16.048880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:16.098144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a31eac559e4ada9bdf241b7dd95aab', 'request_id': 'req-5915069d-be91-4141-82a8-1e1fda5b4225'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.105892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a31eac559e4ada9bdf241b7dd95aab', 'request_id': 'req-5915069d-be91-4141-82a8-1e1fda5b4225'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.114914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a31eac559e4ada9bdf241b7dd95aab', 'request_id': 'req-5915069d-be91-4141-82a8-1e1fda5b4225'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.128446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] The token cache is being invalidate because project c6a31eac559e4ada9bdf241b7dd95aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:31:16.128820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.129554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.131117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1282/2569] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:31:16 2022] DELETE /identity/v3/projects/c6a31eac559e4ada9bdf241b7dd95aab => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:31:16.135251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:16.145886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85597f-8354-4050-8911-be2bd12e80f5 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:16.208376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:16.209001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:16.209282 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:16.209558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] PATH_INFO: `/v3/users/e9fe8511ce144954b4156f7174b70c0b` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:16.210712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fe8511ce144954b4156f7174b70c0b)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:16.213504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:16.259548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fe8511ce144954b4156f7174b70c0b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.267016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fe8511ce144954b4156f7174b70c0b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.267708 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fe8511ce144954b4156f7174b70c0b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.276763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fe8511ce144954b4156f7174b70c0b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.284789 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:31:16.291616 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fe8511ce144954b4156f7174b70c0b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.298960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1288/2570] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:31:16 2022] DELETE /identity/v3/users/e9fe8511ce144954b4156f7174b70c0b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:31:16.688490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:16.690110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:16.707667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915069d-be91-4141-82a8-1e1fda5b4225 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:16.773053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:16.773692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:16.774008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:16.774287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] PATH_INFO: `/v3/projects/5a41ece4975047868c03a143201aa4a7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:16.775409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a41ece4975047868c03a143201aa4a7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:16.777354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:16.778882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:31:16.829877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a41ece4975047868c03a143201aa4a7', 'request_id': 'req-71db3fd1-3797-455a-b772-d9cf872cdf92'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.836824 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a41ece4975047868c03a143201aa4a7', 'request_id': 'req-71db3fd1-3797-455a-b772-d9cf872cdf92'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.850497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a41ece4975047868c03a143201aa4a7', 'request_id': 'req-71db3fd1-3797-455a-b772-d9cf872cdf92'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.864066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] The token cache is being invalidate because project 5a41ece4975047868c03a143201aa4a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:31:16.864383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.864989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:16.866487 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1283/2571] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:31:16 2022] DELETE /identity/v3/projects/5a41ece4975047868c03a143201aa4a7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:31:16.870600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:16.872268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:16.885969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e3fb8-f4ce-45ec-86ee-7306caebb508 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:16.944867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:16.945500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:16.945798 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:16.946101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] PATH_INFO: `/v3/users/bbe7d3ceb4fc448c93d72ce46c73ec81` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:16.947209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe7d3ceb4fc448c93d72ce46c73ec81)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:16.949107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:16.999084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe7d3ceb4fc448c93d72ce46c73ec81'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.007345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe7d3ceb4fc448c93d72ce46c73ec81'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.007977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe7d3ceb4fc448c93d72ce46c73ec81'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.015249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe7d3ceb4fc448c93d72ce46c73ec81'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.027561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-eb79ac4b-79bd-4857-b7e7-b8647989fb8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe7d3ceb4fc448c93d72ce46c73ec81'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.033304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1289/2572] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:31:16 2022] DELETE /identity/v3/users/bbe7d3ceb4fc448c93d72ce46c73ec81 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:31:17.326165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:17.327757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:17.341722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db3fd1-3797-455a-b772-d9cf872cdf92 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:17.398198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:17.398834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.399118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.399416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] PATH_INFO: `/v3/projects/79b278a5a4364895a59b5e06bf89c4dd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.400555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79b278a5a4364895a59b5e06bf89c4dd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:17.402594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:17.451750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79b278a5a4364895a59b5e06bf89c4dd', 'request_id': 'req-52e7672e-7274-4414-8011-c25a57a058f9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.457715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79b278a5a4364895a59b5e06bf89c4dd', 'request_id': 'req-52e7672e-7274-4414-8011-c25a57a058f9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.463469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79b278a5a4364895a59b5e06bf89c4dd', 'request_id': 'req-52e7672e-7274-4414-8011-c25a57a058f9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.475951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] The token cache is being invalidate because project 79b278a5a4364895a59b5e06bf89c4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:31:17.476241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.476847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-52e7672e-7274-4414-8011-c25a57a058f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:31:17.478351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1284/2573] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:31:17 2022] DELETE /identity/v3/projects/79b278a5a4364895a59b5e06bf89c4dd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:31:17.483138 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0052872-8711-4d68-aec6-55170f164f26 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.483599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0052872-8711-4d68-aec6-55170f164f26 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.483698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0052872-8711-4d68-aec6-55170f164f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.504209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a0052872-8711-4d68-aec6-55170f164f26 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:31:17.505414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a0052872-8711-4d68-aec6-55170f164f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:17.562328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1290/2574] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:31:17.570289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ba859-944d-4344-afe1-4a4549036dda None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.570602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ba859-944d-4344-afe1-4a4549036dda None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.570876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ba859-944d-4344-afe1-4a4549036dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.590260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-738ba859-944d-4344-afe1-4a4549036dda None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:31:17.591455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-738ba859-944d-4344-afe1-4a4549036dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:17.599647 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1285/2575] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:31:17.603778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0052872-8711-4d68-aec6-55170f164f26 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:17.613662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0052872-8711-4d68-aec6-55170f164f26 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:17.618124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:17.618704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.618980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.619249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.619883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:31:17.620181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:17.621735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:17.628196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:31:17.634223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1291/2576] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 12:31:17.638587 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ba859-944d-4344-afe1-4a4549036dda None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:17.647747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ba859-944d-4344-afe1-4a4549036dda None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:17.652438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:17.653033 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.653316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.653589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.654268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:17.656203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:17.667597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:31:17.685789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1286/2577] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:31:17 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:31:17.690155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:17.700281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8819-b8d8-446a-a8d3-e47127256334 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:17.755595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:17.756188 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.756463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.756733 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.757359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:17.759263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:17.781421 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1292/2578] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:31:17 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:31:17.786246 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:17.796428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3095-73a3-4980-8896-4e002416247d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:17.802506 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:17.803108 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.803395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.803670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.804314 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:17.806318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:17.816313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1287/2579] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:31:17.820217 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:17.830020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81079f-363f-4bcc-8601-8ea1cbbe6c4b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:17.834988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70708fc2-22be-427e-995f-f375bbb53da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:17.835576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70708fc2-22be-427e-995f-f375bbb53da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.835854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70708fc2-22be-427e-995f-f375bbb53da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.836123 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70708fc2-22be-427e-995f-f375bbb53da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.837012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70708fc2-22be-427e-995f-f375bbb53da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:17.838612 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70708fc2-22be-427e-995f-f375bbb53da7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:17.849473 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1293/2580] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:31:17.853428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:31:17.863531 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff046ae-165f-4368-8c3c-1dc88685430f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:31:17.868099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:31:17.868740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.869023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.869295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] PATH_INFO: `/v3/projects/6534567a9fc348d2b94d1b738a55bd4c/users/53aaf5a05edf48fd9edf1a4b6095c743/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.884109 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6534567a9fc348d2b94d1b738a55bd4c, user_id=53aaf5a05edf48fd9edf1a4b6095c743, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:31:17.886796 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:31:17.900434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1288/2581] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:31:17 2022] PUT /identity/v3/projects/6534567a9fc348d2b94d1b738a55bd4c/users/53aaf5a05edf48fd9edf1a4b6095c743/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:31:17.912440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:31:17.912722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:31:17.912995 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:31:17.939491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] MFA Rules not processed for user `53aaf5a05edf48fd9edf1a4b6095c743`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:31:17.940694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:31:18.002739 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1294/2582] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:31:27.896520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=18,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:31:27.999740 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:07.201281 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:07.203219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:07.255249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:07.256778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:07.271694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc61ee-f9a6-449a-857c-4f950f575ee9 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:07.271877 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fea996-95f9-4e71-8808-a3b5e055e5a6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:07.276244 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64973fe7-c55b-4263-8a7e-e778b0763fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:07.276558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:07.277156 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:07.277265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64973fe7-c55b-4263-8a7e-e778b0763fde service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:07.277439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:07.277640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64973fe7-c55b-4263-8a7e-e778b0763fde service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:07.277729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:07.278234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64973fe7-c55b-4263-8a7e-e778b0763fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:07.279836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:07.289123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64973fe7-c55b-4263-8a7e-e778b0763fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:07.290686 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64973fe7-c55b-4263-8a7e-e778b0763fde service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] Could not find project: 2f7dd7174903460989484c1ab233208a.: keystone.exception.ProjectNotFound: Could not find project: 2f7dd7174903460989484c1ab233208a. Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 19 12:32:07.293885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 12:32:07.294831 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 12:32:07.295850 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application with Lock( Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._enter() Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.creator() Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application created_value = creator( Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 12:32:07.296872 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 12:32:07.297744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2f7dd7174903460989484c1ab233208a. Sep 19 12:32:07.297744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: ERROR keystone.server.flask.application Sep 19 12:32:07.297744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1295/2583] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:07 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 19 12:32:07.301580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1289/2584] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:07.321236 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:07.406663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1403e-8eea-41a2-a792-1a412c6f22b2 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:07.413097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:07.413992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:07.414351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:07.414844 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] PATH_INFO: `/v3/users/53aaf5a05edf48fd9edf1a4b6095c743` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:07.416080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53aaf5a05edf48fd9edf1a4b6095c743)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:07.418310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:07.484769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53aaf5a05edf48fd9edf1a4b6095c743'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:07.492900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53aaf5a05edf48fd9edf1a4b6095c743'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:07.493679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53aaf5a05edf48fd9edf1a4b6095c743'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:07.502252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53aaf5a05edf48fd9edf1a4b6095c743'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:07.518267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53aaf5a05edf48fd9edf1a4b6095c743'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:07.526715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1296/2585] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:32:07 2022] DELETE /identity/v3/users/53aaf5a05edf48fd9edf1a4b6095c743 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:07.531944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:07.532215 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:07.532604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:07.556047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:07.557391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:07.628340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1290/2586] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:07.641256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:07.643762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:07.701704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deea8f-631c-4b05-8ded-fd503b4deb07 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:07.706071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ada31a-2f22-4181-abe0-6c7478929c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:07.706676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ada31a-2f22-4181-abe0-6c7478929c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:07.706948 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ada31a-2f22-4181-abe0-6c7478929c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:07.707215 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ada31a-2f22-4181-abe0-6c7478929c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:07.717003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ada31a-2f22-4181-abe0-6c7478929c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:07.718435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ada31a-2f22-4181-abe0-6c7478929c5f service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:07.728035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1297/2587] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:07.995011 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:07.996476 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:08.011817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480191d5-e152-43c0-8dee-c11144fe0ff4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.016432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.017043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.017327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.017594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] PATH_INFO: `/v3/projects/6534567a9fc348d2b94d1b738a55bd4c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.018740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6534567a9fc348d2b94d1b738a55bd4c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.020666 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.076606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6534567a9fc348d2b94d1b738a55bd4c', 'request_id': 'req-9bf2fbde-d961-4310-883f-46c81228d84a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:08.083944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6534567a9fc348d2b94d1b738a55bd4c', 'request_id': 'req-9bf2fbde-d961-4310-883f-46c81228d84a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:08.090287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6534567a9fc348d2b94d1b738a55bd4c', 'request_id': 'req-9bf2fbde-d961-4310-883f-46c81228d84a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:08.103367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] The token cache is being invalidate because project 6534567a9fc348d2b94d1b738a55bd4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:32:08.103657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:08.104388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9bf2fbde-d961-4310-883f-46c81228d84a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:08.106124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1291/2588] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:32:07 2022] DELETE /identity/v3/projects/6534567a9fc348d2b94d1b738a55bd4c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:08.111661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ab32-650c-4003-805a-f179d96bbc37 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.112142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ab32-650c-4003-805a-f179d96bbc37 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.112594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ab32-650c-4003-805a-f179d96bbc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.140070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d1e4ab32-650c-4003-805a-f179d96bbc37 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:08.141743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d1e4ab32-650c-4003-805a-f179d96bbc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:08.156901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:08.235212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1298/2589] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:08.242290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93c24-eaeb-4923-9283-7a36134189b7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.244005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93c24-eaeb-4923-9283-7a36134189b7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.244697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93c24-eaeb-4923-9283-7a36134189b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.282941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-09e93c24-eaeb-4923-9283-7a36134189b7 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:08.284195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09e93c24-eaeb-4923-9283-7a36134189b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:08.293811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1292/2590] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:08.297355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4ab32-650c-4003-805a-f179d96bbc37 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.308280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4ab32-650c-4003-805a-f179d96bbc37 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.320002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.320637 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.320920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.321194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.321896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:32:08.322207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.323817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.337319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1299/2591] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:32:08.341847 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93c24-eaeb-4923-9283-7a36134189b7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.352381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93c24-eaeb-4923-9283-7a36134189b7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.361058 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.362354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.362631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.362896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.363660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.365708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.394131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:08.401975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1293/2592] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:32:08 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:32:08.406709 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.423316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99642bc-c20b-48ed-956c-776c2ea0a2f1 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.544891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.547504 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.547922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.548047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.551959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.551959 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.599163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1300/2593] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:32:08 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:32:08.599676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.612914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c727f7e-c0cb-4c3d-b635-f2871f34a90f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.619659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.620581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.621110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.621561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.622493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.624422 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.653812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1294/2594] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:32:08.657620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.669925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931d40a-9716-4e71-902a-86adbcff9a29 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.675550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d4be2-33b1-4c97-bff5-1e28fc1ca21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.677074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4be2-33b1-4c97-bff5-1e28fc1ca21f admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.677213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4be2-33b1-4c97-bff5-1e28fc1ca21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.677400 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4be2-33b1-4c97-bff5-1e28fc1ca21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.678089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d4be2-33b1-4c97-bff5-1e28fc1ca21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.679993 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d4be2-33b1-4c97-bff5-1e28fc1ca21f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.692618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1301/2595] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:32:08.696803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.710769 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9deec-5722-406e-a197-9fa90acd64a1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.716360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.717154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.717623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.717936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] PATH_INFO: `/v3/projects/97dfe8f8c7f34999964eaa0e2f954c12/users/c1dd713485744b6da5e24d3c4c84de6a/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.737113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dfe8f8c7f34999964eaa0e2f954c12, user_id=c1dd713485744b6da5e24d3c4c84de6a, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.741045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.756190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1295/2596] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:32:08 2022] PUT /identity/v3/projects/97dfe8f8c7f34999964eaa0e2f954c12/users/c1dd713485744b6da5e24d3c4c84de6a/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:08.769954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3b6ec-b37f-40df-9c10-130a3a2bb655 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.770486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3b6ec-b37f-40df-9c10-130a3a2bb655 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.770954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3b6ec-b37f-40df-9c10-130a3a2bb655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.803938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-6ef3b6ec-b37f-40df-9c10-130a3a2bb655 None None] MFA Rules not processed for user `c1dd713485744b6da5e24d3c4c84de6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:08.805181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6ef3b6ec-b37f-40df-9c10-130a3a2bb655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:08.871135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1302/2597] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:08.875530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.935589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4feee4-10f5-4b05-b4f9-7b6c9f8decc5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.940163 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:08.940765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:08.941043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:08.941313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:08.941963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:08.943841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:08.970537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1296/2598] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:32:08 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:32:08.975108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3b6ec-b37f-40df-9c10-130a3a2bb655 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:08.985540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3b6ec-b37f-40df-9c10-130a3a2bb655 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:08.990541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:09.047311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.047924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.048205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.048478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.049144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.051075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.075775 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1303/2599] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:32:08 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:32:09.080444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.091260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b91e8-a078-4576-8e0b-7be880e1fc4d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.096496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.097095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.097376 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.097651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.098318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.099898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.110494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1297/2600] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:32:09.114579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.124811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac10086-dfa3-4b9c-8441-06a46065686d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.130021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a164ba1-d59c-4bda-99fa-d75d4a8a9aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.130624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164ba1-d59c-4bda-99fa-d75d4a8a9aec admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.130901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164ba1-d59c-4bda-99fa-d75d4a8a9aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.131294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164ba1-d59c-4bda-99fa-d75d4a8a9aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.131947 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a164ba1-d59c-4bda-99fa-d75d4a8a9aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.133539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a164ba1-d59c-4bda-99fa-d75d4a8a9aec admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.145723 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1304/2601] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:32:09.150099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.161492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d322d-a4c1-4e7d-b691-7c9c36547002 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.166275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.166924 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.167208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.167496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] PATH_INFO: `/v3/projects/11363424db794353ae59f04fee2b3d19/users/287e42694d904da0bd574a0cac826d39/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.184258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11363424db794353ae59f04fee2b3d19, user_id=287e42694d904da0bd574a0cac826d39, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.187501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.200884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1298/2602] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:32:09 2022] PUT /identity/v3/projects/11363424db794353ae59f04fee2b3d19/users/287e42694d904da0bd574a0cac826d39/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:09.212811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47e870-1709-4b62-b625-e9fc4577347d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.213098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47e870-1709-4b62-b625-e9fc4577347d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.213374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47e870-1709-4b62-b625-e9fc4577347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.240393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-3b47e870-1709-4b62-b625-e9fc4577347d None None] MFA Rules not processed for user `287e42694d904da0bd574a0cac826d39`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:09.241594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-3b47e870-1709-4b62-b625-e9fc4577347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:09.306627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1305/2603] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:09.311165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.372579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8234ab-427f-4993-bff5-c0922741df3b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.377788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.378340 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.378619 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.378892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.379515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.381395 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.401172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:09.408662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1299/2604] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:32:09 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:32:09.412692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47e870-1709-4b62-b625-e9fc4577347d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.423243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47e870-1709-4b62-b625-e9fc4577347d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.479041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.479651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.480038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.480225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.480827 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.482766 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.507478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1306/2605] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:32:09 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:32:09.511584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.523166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06a15-cb41-4b5f-9849-1769d0024881 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.528620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.529291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.529660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.530028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.530748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.532410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.546984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1300/2606] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:32:09.548290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.558829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15ecbb-64ae-4b02-a4aa-21e3157e0735 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.565178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.565865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.566368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.566652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] PATH_INFO: `/v3/projects/2102388f5e714411a3ae30148e19aea7/users/a7ca815277ba4e73a6824937152c1b08/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.568997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:09.583550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2102388f5e714411a3ae30148e19aea7, user_id=a7ca815277ba4e73a6824937152c1b08, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.586237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.598748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1307/2607] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:32:09 2022] PUT /identity/v3/projects/2102388f5e714411a3ae30148e19aea7/users/a7ca815277ba4e73a6824937152c1b08/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:09.602711 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.613307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2359d-9a7e-4146-9e3a-0724a2aae795 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.672994 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cd543-605a-4006-87bc-c9309bccc80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.673515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cd543-605a-4006-87bc-c9309bccc80f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.674381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cd543-605a-4006-87bc-c9309bccc80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.674645 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cd543-605a-4006-87bc-c9309bccc80f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.675275 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cd543-605a-4006-87bc-c9309bccc80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.676935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cd543-605a-4006-87bc-c9309bccc80f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.687590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1301/2608] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:32:09.692469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.702873 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61ae27-03f4-4cbe-9a90-0f09271b20bf admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.707467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.708112 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.708422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.708695 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] PATH_INFO: `/v3/projects/2102388f5e714411a3ae30148e19aea7/users/a7ca815277ba4e73a6824937152c1b08/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.710627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2102388f5e714411a3ae30148e19aea7, user_id=a7ca815277ba4e73a6824937152c1b08, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.713476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.723674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1308/2609] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:32:09 2022] PUT /identity/v3/projects/2102388f5e714411a3ae30148e19aea7/users/a7ca815277ba4e73a6824937152c1b08/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:09.733843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.734131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.734432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.762934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] MFA Rules not processed for user `a7ca815277ba4e73a6824937152c1b08`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:09.764173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:09.829746 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1302/2610] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:09.841429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.843227 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:09.906035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88dc77-7668-4fc1-92ed-e0270577abcd admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.910054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:09.910699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:09.910929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:09.911199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:09.934593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:09.936069 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:09.978126 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:09.979535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:09.989596 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1309/2611] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:09.995843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90733c88-4d84-42ea-9748-c1d47a77d0dd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:09.999651 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:10.000257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:10.000541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:10.000811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:10.003230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:10.029631 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:10.079203 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:10.080836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:10.090657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1303/2612] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:32:11.535269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:11.537070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:11.548375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b3dcd-a869-4bf4-b3d9-a11ef6fba680 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:11.552493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:11.553211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:11.553496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:11.553804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:11.612423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:11.613871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:11.622832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1310/2613] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:12.324943 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:12.334671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed0b4c-b9ba-46c9-8ed5-094881386d66 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:12.338727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:12.339321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:12.339617 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:12.339909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:12.350239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:12.351649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:12.362492 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1304/2614] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:32:12.996814 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:12.998330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.010617 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9be5ec-4036-4810-8840-fd2f6903ee79 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:13.014201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:13.016855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:13.019239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:13.019601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:13.023299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.039627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:13.040996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:13.052098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1311/2615] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:13.309731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:13.311296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.399805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007631f7-9d8b-4d4a-a46d-4763aca0cda8 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:13.410870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:13.411020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:13.411604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:13.411891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:13.412171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:13.413293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.414582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.428315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff67fb6-bbe7-436b-b5f0-a9ab18ab6d5c service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:13.430000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:13.432051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:13.434347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:13.435476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:13.435953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:13.436444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] PATH_INFO: `/v3/users/766511b86ad643c099998fc4e1638f9a` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:13.437719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=766511b86ad643c099998fc4e1638f9a)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:13.440652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:13.443800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:13.448514 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1305/2616] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:13.539033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766511b86ad643c099998fc4e1638f9a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.557341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766511b86ad643c099998fc4e1638f9a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.558359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766511b86ad643c099998fc4e1638f9a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.571480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766511b86ad643c099998fc4e1638f9a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.589322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766511b86ad643c099998fc4e1638f9a'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.598208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1312/2617] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:32:13 2022] DELETE /identity/v3/users/766511b86ad643c099998fc4e1638f9a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:32:13.637418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:13.662195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68183f00-98d4-4f98-b6c9-e3bb8a57bc60 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:13.725434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:13.727164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.779925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:13.780590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:13.780869 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:13.781188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:13.783800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.796509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:13.798406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:13.817983 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1306/2618] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:13.832083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:13.833798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:13.857177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39435-df77-4a95-aa24-93f3f99b6565 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:13.867103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:13.867898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:13.868184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:13.868499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] PATH_INFO: `/v3/users/828f876065e4413f9447efb451cb5371` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:13.870219 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828f876065e4413f9447efb451cb5371)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:13.872355 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:13.888361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e5370-23e6-4cc0-b2f3-f04e21e4b3eb service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:13.899565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:13.901563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:13.902161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:13.902642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] PATH_INFO: `/v3/projects/256ccce56c534cacb247aa128399f3f1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:13.904025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=256ccce56c534cacb247aa128399f3f1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:13.906090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:13.937919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:13.947288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828f876065e4413f9447efb451cb5371'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.958260 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828f876065e4413f9447efb451cb5371'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.959044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828f876065e4413f9447efb451cb5371'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.968648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828f876065e4413f9447efb451cb5371'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.971871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256ccce56c534cacb247aa128399f3f1', 'request_id': 'req-11892eb2-cb99-4607-8852-9d4c8807cc16'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.978859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256ccce56c534cacb247aa128399f3f1', 'request_id': 'req-11892eb2-cb99-4607-8852-9d4c8807cc16'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.985678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-a3d46b26-93f2-4808-9c3e-8cacf9696a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828f876065e4413f9447efb451cb5371'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.987160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256ccce56c534cacb247aa128399f3f1', 'request_id': 'req-11892eb2-cb99-4607-8852-9d4c8807cc16'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:13.989393 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:13.995926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1313/2619] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:32:13 2022] DELETE /identity/v3/users/828f876065e4413f9447efb451cb5371 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:14.004284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] The token cache is being invalidate because project 256ccce56c534cacb247aa128399f3f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:32:14.004615 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:14.005397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-11892eb2-cb99-4607-8852-9d4c8807cc16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:14.007214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1307/2620] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:32:13 2022] DELETE /identity/v3/projects/256ccce56c534cacb247aa128399f3f1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:32:14.011819 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aad1c-d7b9-47e7-94ee-18f84eca301f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.012373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aad1c-d7b9-47e7-94ee-18f84eca301f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.012373 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aad1c-d7b9-47e7-94ee-18f84eca301f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.032173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a08aad1c-d7b9-47e7-94ee-18f84eca301f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:14.034096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a08aad1c-d7b9-47e7-94ee-18f84eca301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:14.107026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1314/2621] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:14.114607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275df233-3d2b-4abf-bae4-72f6e99c5591 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.114843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275df233-3d2b-4abf-bae4-72f6e99c5591 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.115114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275df233-3d2b-4abf-bae4-72f6e99c5591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.137212 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-275df233-3d2b-4abf-bae4-72f6e99c5591 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:14.137519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-275df233-3d2b-4abf-bae4-72f6e99c5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:14.150170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1308/2622] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:14.151368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aad1c-d7b9-47e7-94ee-18f84eca301f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.163389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aad1c-d7b9-47e7-94ee-18f84eca301f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.169085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.169715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.170047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.170326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.171567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:32:14.171567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.172900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.191945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1315/2623] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:32:14.196898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275df233-3d2b-4abf-bae4-72f6e99c5591 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.207975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.209741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:14.209891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275df233-3d2b-4abf-bae4-72f6e99c5591 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.215481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.216090 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.216368 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.216638 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.217261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.219170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.227639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e618275-2042-4bc0-b137-b5e993be5834 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.234398 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.235322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.236047 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.236330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] PATH_INFO: `/v3/projects/b0aec947283643dd9dfdce33a7b2799e` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.237690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0aec947283643dd9dfdce33a7b2799e)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.239838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.249451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1309/2624] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:32:14 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:32:14.253611 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.290315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0aec947283643dd9dfdce33a7b2799e', 'request_id': 'req-54c3b1fe-5422-4403-bfd1-e84622d40fff'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:14.298258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0aec947283643dd9dfdce33a7b2799e', 'request_id': 'req-54c3b1fe-5422-4403-bfd1-e84622d40fff'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:14.304970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0aec947283643dd9dfdce33a7b2799e', 'request_id': 'req-54c3b1fe-5422-4403-bfd1-e84622d40fff'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:14.315301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] The token cache is being invalidate because project b0aec947283643dd9dfdce33a7b2799e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:32:14.315565 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:14.316281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:14.317837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1316/2625] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:32:14 2022] DELETE /identity/v3/projects/b0aec947283643dd9dfdce33a7b2799e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:14.318602 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981b256-ebdf-47e0-a577-876b67ea3ade admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.384381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.385004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.385312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.385590 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.386265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.388202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.417393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1310/2626] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:32:14 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:32:14.419348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.420863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:14.434149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3b1fe-5422-4403-bfd1-e84622d40fff admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.439006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.440124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.440401 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.440670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.441344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.442997 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.454179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1317/2627] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:32:14.459155 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.469973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20863587-6616-43c8-9eb1-e05a26f8422c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.474729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1f35d-0427-480d-8bfa-0b82b608b83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.475361 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f35d-0427-480d-8bfa-0b82b608b83c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.475660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f35d-0427-480d-8bfa-0b82b608b83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.475915 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f35d-0427-480d-8bfa-0b82b608b83c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.476591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1f35d-0427-480d-8bfa-0b82b608b83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.478263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1f35d-0427-480d-8bfa-0b82b608b83c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.489232 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1311/2628] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:32:14.493691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.504435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e948a09-58e9-4d57-9e77-bef3b11e2d48 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.509186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.509889 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.510166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.510435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] PATH_INFO: `/v3/projects/3da2a386a837446dad4bf8b8506e418d/users/f4fd2deca6124518ae1df9ed7977f350/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.526577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da2a386a837446dad4bf8b8506e418d, user_id=f4fd2deca6124518ae1df9ed7977f350, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.529192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.542071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1318/2629] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:32:14 2022] PUT /identity/v3/projects/3da2a386a837446dad4bf8b8506e418d/users/f4fd2deca6124518ae1df9ed7977f350/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:32:14.547057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.547346 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.547622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.555705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:14.568954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:14.570185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:14.628260 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1312/2630] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:14.638014 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:14.640560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:14.685342 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:14.704597 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63ef0a-ee31-4b3d-bdfb-81a45f19333f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:14.708912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed931846-0929-4301-a195-0e7f1cd743f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:14.709513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931846-0929-4301-a195-0e7f1cd743f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:14.709820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931846-0929-4301-a195-0e7f1cd743f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:14.710097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931846-0929-4301-a195-0e7f1cd743f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:14.719220 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed931846-0929-4301-a195-0e7f1cd743f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:14.720635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed931846-0929-4301-a195-0e7f1cd743f6 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:14.730004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1319/2631] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:16.452932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:16.454441 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:16.470706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ca3de-a2f0-474b-9dad-228569c15a8c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:16.476922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:16.477569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:16.477874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:16.478145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] PATH_INFO: `/v3/users/c1dd713485744b6da5e24d3c4c84de6a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:16.479379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1dd713485744b6da5e24d3c4c84de6a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:16.481285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:16.539156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1dd713485744b6da5e24d3c4c84de6a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:16.539832 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1dd713485744b6da5e24d3c4c84de6a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:16.547068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1dd713485744b6da5e24d3c4c84de6a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:16.555974 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1dd713485744b6da5e24d3c4c84de6a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:16.570085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1dd713485744b6da5e24d3c4c84de6a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:16.578097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1313/2632] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:32:16 2022] DELETE /identity/v3/users/c1dd713485744b6da5e24d3c4c84de6a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:16.582600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:16.583055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:16.583308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:16.604656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:16.605899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:16.671714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1320/2633] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:16.682186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:16.683834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:16.752971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077331-5728-40af-aa7b-b8ef1787b7af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:16.759477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:16.760123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:16.760912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:16.760912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:16.774693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:16.777269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:16.791291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1314/2634] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:17.121625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:17.123212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:17.137150 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975abb1f-1320-4c05-a61e-716d1860a35b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:17.150611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:17.151370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:17.151875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:17.152181 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] PATH_INFO: `/v3/projects/97dfe8f8c7f34999964eaa0e2f954c12` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:17.153449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97dfe8f8c7f34999964eaa0e2f954c12)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:17.155883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:17.221475 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97dfe8f8c7f34999964eaa0e2f954c12', 'request_id': 'req-e46176ca-1081-4c82-8034-a57c6d681585'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.231493 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97dfe8f8c7f34999964eaa0e2f954c12', 'request_id': 'req-e46176ca-1081-4c82-8034-a57c6d681585'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.240812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97dfe8f8c7f34999964eaa0e2f954c12', 'request_id': 'req-e46176ca-1081-4c82-8034-a57c6d681585'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.256218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] The token cache is being invalidate because project 97dfe8f8c7f34999964eaa0e2f954c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:32:17.257312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.258238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.260897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1321/2635] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:32:17 2022] DELETE /identity/v3/projects/97dfe8f8c7f34999964eaa0e2f954c12 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:17.264123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:17.280643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a59a6-b751-4432-b2a0-d83dd7554b7e service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:17.301511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:17.353929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:17.354481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:17.354779 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:17.355042 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] PATH_INFO: `/v3/users/287e42694d904da0bd574a0cac826d39` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:17.356799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=287e42694d904da0bd574a0cac826d39)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:17.358821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:17.416659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '287e42694d904da0bd574a0cac826d39'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.417715 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '287e42694d904da0bd574a0cac826d39'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.427961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '287e42694d904da0bd574a0cac826d39'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.442305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '287e42694d904da0bd574a0cac826d39'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.458207 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '287e42694d904da0bd574a0cac826d39'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:17.467143 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1315/2636] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:32:17 2022] DELETE /identity/v3/users/287e42694d904da0bd574a0cac826d39 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:17.817000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:17.818469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:17.834003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46176ca-1081-4c82-8034-a57c6d681585 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:17.970249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:17.971026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:17.971450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:17.972087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] PATH_INFO: `/v3/projects/11363424db794353ae59f04fee2b3d19` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:17.972914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11363424db794353ae59f04fee2b3d19)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:17.975202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:17.978054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:18.056149 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11363424db794353ae59f04fee2b3d19', 'request_id': 'req-abed2ceb-576c-4da2-969f-997f7830f869'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.076567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11363424db794353ae59f04fee2b3d19', 'request_id': 'req-abed2ceb-576c-4da2-969f-997f7830f869'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.085836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11363424db794353ae59f04fee2b3d19', 'request_id': 'req-abed2ceb-576c-4da2-969f-997f7830f869'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.103168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] The token cache is being invalidate because project 11363424db794353ae59f04fee2b3d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:32:18.103168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.103815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.106620 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1322/2637] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:32:17 2022] DELETE /identity/v3/projects/11363424db794353ae59f04fee2b3d19 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:18.109985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:18.111650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:18.128085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa7ff-d8b1-44c3-a991-8e5ea9bfaf4b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:18.233718 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:18.235228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:18.235562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:18.235885 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] PATH_INFO: `/v3/users/a7ca815277ba4e73a6824937152c1b08` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:18.237656 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ca815277ba4e73a6824937152c1b08)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:18.241174 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:18.320818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:18.329342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ca815277ba4e73a6824937152c1b08'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.331284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ca815277ba4e73a6824937152c1b08'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.341558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ca815277ba4e73a6824937152c1b08'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.352909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ca815277ba4e73a6824937152c1b08'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.368938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544122ca-eeab-4399-af83-bfd156174b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ca815277ba4e73a6824937152c1b08'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:18.380863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1316/2638] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:32:18 2022] DELETE /identity/v3/users/a7ca815277ba4e73a6824937152c1b08 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:18.832891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:18.834454 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:18.857382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2ceb-576c-4da2-969f-997f7830f869 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:18.944946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:18.945615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:18.945939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:18.946295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] PATH_INFO: `/v3/projects/2102388f5e714411a3ae30148e19aea7` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:18.947705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2102388f5e714411a3ae30148e19aea7)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:18.949432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.036020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2102388f5e714411a3ae30148e19aea7', 'request_id': 'req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:19.045749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2102388f5e714411a3ae30148e19aea7', 'request_id': 'req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:19.054016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2102388f5e714411a3ae30148e19aea7', 'request_id': 'req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:19.066694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.067195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.067367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.067885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] The token cache is being invalidate because project 2102388f5e714411a3ae30148e19aea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:32:19.068247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:19.068915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-8af9d2f1-152c-49fe-8dab-8cc03d47c95e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:32:19.070842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1323/2639] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:32:18 2022] DELETE /identity/v3/projects/2102388f5e714411a3ae30148e19aea7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:32:19.075437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4262ea7-db9e-4ee9-9625-a1b6e0bcfe1a None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.075728 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4262ea7-db9e-4ee9-9625-a1b6e0bcfe1a None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.076003 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4262ea7-db9e-4ee9-9625-a1b6e0bcfe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.096659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d4262ea7-db9e-4ee9-9625-a1b6e0bcfe1a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:19.097913 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d4262ea7-db9e-4ee9-9625-a1b6e0bcfe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:19.105609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] MFA Rules not processed for user `f4fd2deca6124518ae1df9ed7977f350`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:19.106876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:19.117143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:19.177118 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1324/2640] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:19.186958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e49cd-cbbe-4721-bc55-a470cf4b8a0c None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.187343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e49cd-cbbe-4721-bc55-a470cf4b8a0c None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.187687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e49cd-cbbe-4721-bc55-a470cf4b8a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.213952 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-f11e49cd-cbbe-4721-bc55-a470cf4b8a0c None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:19.215274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f11e49cd-cbbe-4721-bc55-a470cf4b8a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:19.227198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1325/2641] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:19.231420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e49cd-cbbe-4721-bc55-a470cf4b8a0c None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.241273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e49cd-cbbe-4721-bc55-a470cf4b8a0c None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.241655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1317/2642] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:19.246911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.247532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.247816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.248088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.248848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:32:19.249159 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.250821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.257142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.259241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:19.264330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1326/2643] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 12:32:19.269012 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.279984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc35e3-097d-4503-a699-dc0b1b2f4a89 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.285624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.286303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.286583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.286857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.287495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.291511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.320157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1327/2644] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:32:19 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:32:19.324785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.353167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25e2f6-0c60-4ae7-b2f5-62d30b8e1c44 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.399299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5f33-daf0-4a04-8040-ed135e66b811 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.404145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.404758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.405039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.405311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.405542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:19.405981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.408554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.430671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a63fb0-dd64-46b1-b64b-919cde81ed5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.431169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1328/2645] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:32:19 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:32:19.431809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a63fb0-dd64-46b1-b64b-919cde81ed5b service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.432272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a63fb0-dd64-46b1-b64b-919cde81ed5b service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.432548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a63fb0-dd64-46b1-b64b-919cde81ed5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.435230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.445461 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa4462-c39c-4b30-9a6d-bc285d10e88f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.450343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.450957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.451238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.451517 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.452190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.453783 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.469108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1329/2646] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:32:19.471043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.481224 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a379b08-09b7-40aa-a5d9-1d6237f34778 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.486061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.486752 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.487101 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.487453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.488174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.489876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.503065 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1330/2647] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:32:19.505074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a63fb0-dd64-46b1-b64b-919cde81ed5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.506646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.506771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a63fb0-dd64-46b1-b64b-919cde81ed5b service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.516686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec560479-81fc-4346-8e62-2f7f6ed77851 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.517944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1318/2648] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:32:19.522991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.523198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.523480 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.523763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] PATH_INFO: `/v3/projects/4c2ce53e7c5a4ccfb823e3873501b42d/users/64ae73c1701343b284fe92b4b4226a3b/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.540451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2ce53e7c5a4ccfb823e3873501b42d, user_id=64ae73c1701343b284fe92b4b4226a3b, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.543529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.569180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1331/2649] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:32:19 2022] PUT /identity/v3/projects/4c2ce53e7c5a4ccfb823e3873501b42d/users/64ae73c1701343b284fe92b4b4226a3b/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:32:19.575057 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec57ff-03ea-4b65-a542-6d4f20733230 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.575342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec57ff-03ea-4b65-a542-6d4f20733230 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.575662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec57ff-03ea-4b65-a542-6d4f20733230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.610781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dcec57ff-03ea-4b65-a542-6d4f20733230 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:19.616436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dcec57ff-03ea-4b65-a542-6d4f20733230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:19.721700 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1319/2650] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:19.731563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:19.733481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:19.765721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:19.804410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f537b0-838f-4bc5-9610-86b0f4de2344 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:19.815611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:19.816725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:19.816725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:19.816918 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:19.828184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:19.830420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:19.841843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1332/2651] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:32:20.733312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d72466-dc82-4660-9e40-5b57a4896985 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:20.733798 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d72466-dc82-4660-9e40-5b57a4896985 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:20.734135 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d72466-dc82-4660-9e40-5b57a4896985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:20.767494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-17d72466-dc82-4660-9e40-5b57a4896985 None None] MFA Rules not processed for user `64ae73c1701343b284fe92b4b4226a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:20.769046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-17d72466-dc82-4660-9e40-5b57a4896985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:20.843496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1320/2652] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:20.855171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:20.858335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:20.936212 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdde31-42db-4122-83b6-4ce2c0b86678 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:20.940095 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:20.940898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:20.941345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:20.941601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:20.952999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:20.955266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:20.967253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1333/2653] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:21.395034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:21.395311 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:21.395581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:21.414975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] MFA Rules not processed for user `f4fd2deca6124518ae1df9ed7977f350`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:21.416186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:21.424925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:21.473524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1321/2654] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:21.790330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:21.791713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:21.803915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281de28-1981-4409-8346-2d2d4b9bfa5c service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:21.807685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c5e7d-7f13-4e38-ab54-5b2d63f51dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:21.808399 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5e7d-7f13-4e38-ab54-5b2d63f51dea service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:21.808644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5e7d-7f13-4e38-ab54-5b2d63f51dea service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:21.808818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5e7d-7f13-4e38-ab54-5b2d63f51dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:21.810487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0b7c5e7d-7f13-4e38-ab54-5b2d63f51dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:21.820900 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c5e7d-7f13-4e38-ab54-5b2d63f51dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:21.822439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c5e7d-7f13-4e38-ab54-5b2d63f51dea service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:21.830605 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1334/2655] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:22.181811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:22.189220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:22.282245 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081cc23-7aef-47d0-923b-00d54d28ca03 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:22.288150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:22.288840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:22.289477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:22.289477 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:22.291932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:22.303898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:22.305695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:22.317735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1322/2656] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:22.425041 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567228-93cf-4640-a7b2-41ee3e9dc7c4 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:22.425550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567228-93cf-4640-a7b2-41ee3e9dc7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:22.425657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567228-93cf-4640-a7b2-41ee3e9dc7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:22.445639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-77567228-93cf-4640-a7b2-41ee3e9dc7c4 None None] MFA Rules not processed for user `64ae73c1701343b284fe92b4b4226a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:22.447234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-77567228-93cf-4640-a7b2-41ee3e9dc7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:22.455364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1335/2657] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:32:31.108467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:31.184297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993a491-dd42-491a-b6c1-fae6624056b8 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:31.190852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ebc22-3874-4c01-936c-78e9107e55c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:31.191473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ebc22-3874-4c01-936c-78e9107e55c1 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:31.191760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ebc22-3874-4c01-936c-78e9107e55c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:31.192052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ebc22-3874-4c01-936c-78e9107e55c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:31.193911 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a22ebc22-3874-4c01-936c-78e9107e55c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:31.207511 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ebc22-3874-4c01-936c-78e9107e55c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:31.209092 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ebc22-3874-4c01-936c-78e9107e55c1 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:31.221504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1323/2658] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:32.439784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=50 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:35.210333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77567228-93cf-4640-a7b2-41ee3e9dc7c4 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:35.224738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77567228-93cf-4640-a7b2-41ee3e9dc7c4 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:35.231427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:35.232154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:35.232470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:35.232711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:35.234600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:35.247942 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:35.249965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:35.264059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1336/2659] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:38.174428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39ba52-3af4-4620-8c55-f61497d2a514 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:38.174868 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39ba52-3af4-4620-8c55-f61497d2a514 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:38.174952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39ba52-3af4-4620-8c55-f61497d2a514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:38.194481 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0d39ba52-3af4-4620-8c55-f61497d2a514 None None] MFA Rules not processed for user `64ae73c1701343b284fe92b4b4226a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:38.195676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d39ba52-3af4-4620-8c55-f61497d2a514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:38.202398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1324/2660] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:38.433358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:38.442476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e18d2-c932-4ee5-a004-533fff4a30fe service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:38.446145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:38.446725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:38.446999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:38.447266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:38.448923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:38.458154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:38.459534 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:38.467693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1337/2661] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:32:48.188648 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:48.464254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=12 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:32:58.101211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de816fe-b9cf-4d02-9d48-70806ab2c0ae None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:58.101712 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de816fe-b9cf-4d02-9d48-70806ab2c0ae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:58.101841 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de816fe-b9cf-4d02-9d48-70806ab2c0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:58.162537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3de816fe-b9cf-4d02-9d48-70806ab2c0ae None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:32:58.163975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3de816fe-b9cf-4d02-9d48-70806ab2c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:58.229521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1325/2662] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:32:58.275830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:32:58.277275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:32:58.364923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c8aa6-104d-4e9d-a19d-9fb3240a30ab service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:32:58.369842 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:32:58.370434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:32:58.370711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:32:58.370980 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:32:58.379425 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:32:58.380960 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:32:58.389032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1338/2663] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:32:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:33:08.226803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=44 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:08.387142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=41 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:13.655185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d8f9-2147-43f5-b527-0628aa05b401 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:13.655696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d8f9-2147-43f5-b527-0628aa05b401 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:13.655696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d8f9-2147-43f5-b527-0628aa05b401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:13.684805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-61f1d8f9-2147-43f5-b527-0628aa05b401 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:13.686472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-61f1d8f9-2147-43f5-b527-0628aa05b401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:13.696412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1326/2664] 10.222.0.88 () {58 vars in 967 bytes} [Mon Sep 19 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:13.709438 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:13.720265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bee827-9706-41c8-ad63-6a1a1ff5fb0c service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:13.725157 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:13.725750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:13.726062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:13.726340 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:13.734444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:13.736017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:13.744077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1339/2665] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:33:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:33:14.519271 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1d8f9-2147-43f5-b527-0628aa05b401 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:14.528717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1d8f9-2147-43f5-b527-0628aa05b401 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:14.532696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:14.533293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:14.533571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:14.533879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:14.535601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:14.548953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:14.550800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:14.560923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1327/2666] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:15.892388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:15.903446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61cd5f-e3fc-4b3a-8ede-24fca2208aed service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:15.910731 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:15.911428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:15.911705 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:15.911977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] PATH_INFO: `/v3/users/64ae73c1701343b284fe92b4b4226a3b` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:15.913270 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ae73c1701343b284fe92b4b4226a3b)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:15.915247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:15.966704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ae73c1701343b284fe92b4b4226a3b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:15.975874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ae73c1701343b284fe92b4b4226a3b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:15.976817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ae73c1701343b284fe92b4b4226a3b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:15.985385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ae73c1701343b284fe92b4b4226a3b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:15.998767 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-39c7c25f-59c7-4754-a62b-5eb063f9540c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ae73c1701343b284fe92b4b4226a3b'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:16.004934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1340/2667] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:15 2022] DELETE /identity/v3/users/64ae73c1701343b284fe92b4b4226a3b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:16.196404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:16.207840 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb2d64-3a70-4a63-a4b8-64d855c34d94 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:16.274640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:16.275278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.275557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.275827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] PATH_INFO: `/v3/projects/4c2ce53e7c5a4ccfb823e3873501b42d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.276979 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c2ce53e7c5a4ccfb823e3873501b42d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:16.278926 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:16.326344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c2ce53e7c5a4ccfb823e3873501b42d', 'request_id': 'req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:16.333096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c2ce53e7c5a4ccfb823e3873501b42d', 'request_id': 'req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:16.339757 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c2ce53e7c5a4ccfb823e3873501b42d', 'request_id': 'req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:16.353504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] The token cache is being invalidate because project 4c2ce53e7c5a4ccfb823e3873501b42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:16.353821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:16.354448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0f27fe17-0fa6-4137-a11c-a13ab4c1c83b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:16.355936 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1328/2668] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:16 2022] DELETE /identity/v3/projects/4c2ce53e7c5a4ccfb823e3873501b42d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:16.360823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f66df3-96b0-47c4-a40b-47c1b60d3b77 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.361109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f66df3-96b0-47c4-a40b-47c1b60d3b77 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.361380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f66df3-96b0-47c4-a40b-47c1b60d3b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.380890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-92f66df3-96b0-47c4-a40b-47c1b60d3b77 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:16.382130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-92f66df3-96b0-47c4-a40b-47c1b60d3b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:16.449056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1341/2669] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:33:16.457130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da016-a413-405d-8e92-65a270bde498 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.457294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da016-a413-405d-8e92-65a270bde498 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.457685 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da016-a413-405d-8e92-65a270bde498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.480729 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e20da016-a413-405d-8e92-65a270bde498 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:16.481971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e20da016-a413-405d-8e92-65a270bde498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:16.493385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1329/2670] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:16.497252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f66df3-96b0-47c4-a40b-47c1b60d3b77 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:16.519274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f66df3-96b0-47c4-a40b-47c1b60d3b77 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:16.539532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:16.540166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.542018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.542018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.542018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:33:16.542018 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:16.543383 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:16.557835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1342/2671] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:33:16.560312 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da016-a413-405d-8e92-65a270bde498 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:16.570976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da016-a413-405d-8e92-65a270bde498 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:16.577046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:16.578764 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.578899 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.579097 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.579719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:16.581612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:16.611105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1330/2672] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:16 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:33:16.614912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:16.626665 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceedd3-385f-4ddd-b617-237b938b61bf admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:16.683832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:16.705232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:16.705885 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.706164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.706431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.707063 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:16.708963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:16.743547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1343/2673] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:16 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:33:16.745093 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:16.760443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d8bba-523c-4a56-a578-21a595166ec6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:16.765366 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:16.766527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.766806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.767074 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.767714 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:16.770023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:16.779672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1331/2674] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:16.783601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:16.793485 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11962e7-0666-4dd4-982d-9aa995bc0fe6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:16.799113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2030b60-18e7-4e14-b64a-136362cfc25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:16.800077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2030b60-18e7-4e14-b64a-136362cfc25c admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.800452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2030b60-18e7-4e14-b64a-136362cfc25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.800791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2030b60-18e7-4e14-b64a-136362cfc25c admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.801552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2030b60-18e7-4e14-b64a-136362cfc25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:16.803305 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2030b60-18e7-4e14-b64a-136362cfc25c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:16.814440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1344/2675] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:16.818504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:16.830961 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a676f-37fb-48d3-897e-78eee84b2a00 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:16.836629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:16.837810 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.838332 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.840567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] PATH_INFO: `/v3/projects/7ef33aff3d864a0eab321a82554e5d61/users/043a167d81eb473e90d757bdb497a3dd/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.856469 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef33aff3d864a0eab321a82554e5d61, user_id=043a167d81eb473e90d757bdb497a3dd, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:16.859918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:16.882704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1332/2676] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:16 2022] PUT /identity/v3/projects/7ef33aff3d864a0eab321a82554e5d61/users/043a167d81eb473e90d757bdb497a3dd/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:16.894353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:16.895589 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:16.896094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:16.942469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] MFA Rules not processed for user `043a167d81eb473e90d757bdb497a3dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:16.943614 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:17.029804 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1345/2677] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:17.040963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:17.042774 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:17.053321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:17.126455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ce83e-23a5-4c50-8bd0-b4bcd02550c8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:17.130467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:17.130844 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:17.131124 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:17.131398 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:17.140484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:17.141938 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:17.153966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1333/2678] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:18.091200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:18.092727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:18.176004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3f614-0e32-419f-b40a-b0ef57b44488 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:18.184949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:18.185611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:18.185925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:18.186200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] PATH_INFO: `/v3/users/8fa4b6af47384c7fb26aae3547d02b64` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:18.187375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fa4b6af47384c7fb26aae3547d02b64)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:18.190463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:18.254391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fa4b6af47384c7fb26aae3547d02b64'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.264921 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fa4b6af47384c7fb26aae3547d02b64'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.266332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fa4b6af47384c7fb26aae3547d02b64'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.268551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:18.276396 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fa4b6af47384c7fb26aae3547d02b64'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.289059 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-12355599-989b-46ff-b1d8-8db1c357cb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fa4b6af47384c7fb26aae3547d02b64'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.294986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1346/2679] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:18 2022] DELETE /identity/v3/users/8fa4b6af47384c7fb26aae3547d02b64 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:18.469811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:18.481560 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e2f1b-e4d0-41a6-a299-52e2b3ab776b service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:18.552786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:18.553484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:18.553740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:18.554050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] PATH_INFO: `/v3/projects/ec6875fa35a4462698c06020a0238285` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:18.566159 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6875fa35a4462698c06020a0238285)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:18.569455 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:18.631171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6875fa35a4462698c06020a0238285', 'request_id': 'req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.640175 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6875fa35a4462698c06020a0238285', 'request_id': 'req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.646850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6875fa35a4462698c06020a0238285', 'request_id': 'req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.662385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] The token cache is being invalidate because project ec6875fa35a4462698c06020a0238285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:18.662726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.663426 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-473cf1b7-8596-41e6-96e1-1a4fc1de7d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:18.665160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1334/2680] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:18 2022] DELETE /identity/v3/projects/ec6875fa35a4462698c06020a0238285 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:18.671040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03ba7-3313-48fd-856c-17361f2b79eb None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:18.671385 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03ba7-3313-48fd-856c-17361f2b79eb None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:18.671716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03ba7-3313-48fd-856c-17361f2b79eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:18.694391 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-02c03ba7-3313-48fd-856c-17361f2b79eb None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:18.695606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-02c03ba7-3313-48fd-856c-17361f2b79eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:18.772558 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1347/2681] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:18.780187 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74cbf9-ab7a-49ef-977d-bb50439475ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:18.780520 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74cbf9-ab7a-49ef-977d-bb50439475ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:18.780893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74cbf9-ab7a-49ef-977d-bb50439475ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:18.802114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3e74cbf9-ab7a-49ef-977d-bb50439475ee None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:18.803302 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3e74cbf9-ab7a-49ef-977d-bb50439475ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:18.811027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1335/2682] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:18.815703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c03ba7-3313-48fd-856c-17361f2b79eb None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:18.825077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c03ba7-3313-48fd-856c-17361f2b79eb None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:18.829660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:18.830278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:18.830551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:18.830817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:18.831451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:33:18.831746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:18.833294 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:18.845264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1348/2683] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:33:18.849782 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74cbf9-ab7a-49ef-977d-bb50439475ee None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:18.855404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:18.860449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74cbf9-ab7a-49ef-977d-bb50439475ee None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:18.866427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:18.867048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:18.867661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:18.868004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:18.868793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:18.871250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:18.897664 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1336/2684] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:18 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:18.901965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:18.912370 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b05d6e-32bd-433c-b026-3e74aa6b812a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:18.967406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:18.968009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:18.968293 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:18.968567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:18.969732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:18.971687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:18.992499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1349/2685] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:18 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:18.997446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.007610 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18339926-b244-4f11-8642-9ff576ac7439 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.012195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.012790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.013073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.013345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.014041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.015621 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.026009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1337/2686] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:19.029793 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.040178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038326b-0ffb-4d5c-8bd7-1b3e91bba3f7 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.044702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47f23f-2f15-4393-b0d9-46cb4dbcd1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.045290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47f23f-2f15-4393-b0d9-46cb4dbcd1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.045566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47f23f-2f15-4393-b0d9-46cb4dbcd1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.045869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47f23f-2f15-4393-b0d9-46cb4dbcd1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.046502 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47f23f-2f15-4393-b0d9-46cb4dbcd1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.048672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47f23f-2f15-4393-b0d9-46cb4dbcd1e3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.059013 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1350/2687] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:33:19.063050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.072819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd91a-31ba-4cdf-bdbd-a9cf87e0b62d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.077318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.077985 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.078283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.078557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] PATH_INFO: `/v3/projects/95abb80de0e34a80a5332ba40aa8b2e2/users/9e029f305b48463ab0bfa91e13308896/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.092854 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95abb80de0e34a80a5332ba40aa8b2e2, user_id=9e029f305b48463ab0bfa91e13308896, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.095990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.108665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1338/2688] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:19 2022] PUT /identity/v3/projects/95abb80de0e34a80a5332ba40aa8b2e2/users/9e029f305b48463ab0bfa91e13308896/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:33:19.118909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb182661-e556-423f-95d8-1474761cc447 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.119358 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb182661-e556-423f-95d8-1474761cc447 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.119447 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb182661-e556-423f-95d8-1474761cc447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.146945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-bb182661-e556-423f-95d8-1474761cc447 None None] MFA Rules not processed for user `9e029f305b48463ab0bfa91e13308896`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:19.148140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-bb182661-e556-423f-95d8-1474761cc447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:19.176863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:19.213828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1351/2689] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:19.218315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.277403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dfa51-7cc8-49a3-a647-7e172677a77c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.283013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.283692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.283975 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.284250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.284891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.286827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.312951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1339/2690] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:19 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:19.316969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb182661-e556-423f-95d8-1474761cc447 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.327359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb182661-e556-423f-95d8-1474761cc447 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.385281 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.386031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.386317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.386615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.387257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.389180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.412626 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1352/2691] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:19 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:33:19.416562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.427133 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b195c-3ebf-4cbf-9043-d79aa951c3ba admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.431706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.432309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.432589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.432863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.433503 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.435125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.446130 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1340/2692] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:33:19.450040 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.460307 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641410-c4e3-4d0e-ba0f-d63431668d01 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.465151 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.465823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.466109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.466382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] PATH_INFO: `/v3/projects/6b3daae5417948858e9b9af256081d0a/users/06b76970e8bd45d5909aa4fe9a86fe57/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.481754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3daae5417948858e9b9af256081d0a, user_id=06b76970e8bd45d5909aa4fe9a86fe57, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.484427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.497625 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1353/2693] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:19 2022] PUT /identity/v3/projects/6b3daae5417948858e9b9af256081d0a/users/06b76970e8bd45d5909aa4fe9a86fe57/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:33:19.501571 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.511931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824c634-9ebc-400d-a2ac-60eafe82717d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.567248 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30ecb5-5be9-4a14-bc91-bf1dacf05fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.567771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ecb5-5be9-4a14-bc91-bf1dacf05fae admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.568052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ecb5-5be9-4a14-bc91-bf1dacf05fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.568379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ecb5-5be9-4a14-bc91-bf1dacf05fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.569023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30ecb5-5be9-4a14-bc91-bf1dacf05fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.570643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30ecb5-5be9-4a14-bc91-bf1dacf05fae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.581007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1341/2694] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:19.585177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.595518 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d534c1-3df2-40ca-a324-606f8275a751 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.600030 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.600664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.600940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.601211 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] PATH_INFO: `/v3/projects/6b3daae5417948858e9b9af256081d0a/users/06b76970e8bd45d5909aa4fe9a86fe57/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.603115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3daae5417948858e9b9af256081d0a, user_id=06b76970e8bd45d5909aa4fe9a86fe57, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.605707 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.615661 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1354/2695] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:19 2022] PUT /identity/v3/projects/6b3daae5417948858e9b9af256081d0a/users/06b76970e8bd45d5909aa4fe9a86fe57/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:19.627066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c27ba-a65c-4e56-82b1-c914d17ec9d1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.627354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c27ba-a65c-4e56-82b1-c914d17ec9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.627640 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c27ba-a65c-4e56-82b1-c914d17ec9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.655485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f77c27ba-a65c-4e56-82b1-c914d17ec9d1 None None] MFA Rules not processed for user `06b76970e8bd45d5909aa4fe9a86fe57`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:19.656727 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f77c27ba-a65c-4e56-82b1-c914d17ec9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:19.661662 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:19.726790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1342/2696] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:19.738134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.739874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:19.798698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef85bd-554a-4954-91e4-a431b39b355e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.802429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd8acd-2562-4945-a890-864b52d315c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.803033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd8acd-2562-4945-a890-864b52d315c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.803313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd8acd-2562-4945-a890-864b52d315c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.803585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd8acd-2562-4945-a890-864b52d315c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.813141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd8acd-2562-4945-a890-864b52d315c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.814754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd8acd-2562-4945-a890-864b52d315c0 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:19.824642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1355/2697] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:19.889850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c27ba-a65c-4e56-82b1-c914d17ec9d1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:19.951490 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c27ba-a65c-4e56-82b1-c914d17ec9d1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:19.956006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:19.956637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:19.956920 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:19.957194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] PATH_INFO: `/v3/users/9e029f305b48463ab0bfa91e13308896` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:19.958347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e029f305b48463ab0bfa91e13308896)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:19.960253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:20.008770 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e029f305b48463ab0bfa91e13308896'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.009401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e029f305b48463ab0bfa91e13308896'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.016131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e029f305b48463ab0bfa91e13308896'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.023962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e029f305b48463ab0bfa91e13308896'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.035996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e029f305b48463ab0bfa91e13308896'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.041897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1343/2698] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:19 2022] DELETE /identity/v3/users/9e029f305b48463ab0bfa91e13308896 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:20.046490 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:20.047301 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:20.047583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:20.066457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:20.067654 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:20.085994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:20.127148 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1356/2699] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:20.136859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:20.138310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:20.164409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:20.196570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f5e5f-fdbb-4532-a90a-485acac9c677 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:20.200189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:20.200787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:20.201066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:20.201338 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:20.211603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:20.213012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:20.222209 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1344/2700] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:20.496253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:20.497866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:20.510084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933ada8-e821-4dde-b5d3-16e9a4dbe232 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:20.514849 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:20.515487 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:20.515770 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:20.516045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] PATH_INFO: `/v3/projects/95abb80de0e34a80a5332ba40aa8b2e2` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:20.517174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95abb80de0e34a80a5332ba40aa8b2e2)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:20.519119 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:20.566412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95abb80de0e34a80a5332ba40aa8b2e2', 'request_id': 'req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.573543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95abb80de0e34a80a5332ba40aa8b2e2', 'request_id': 'req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.579890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95abb80de0e34a80a5332ba40aa8b2e2', 'request_id': 'req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.590407 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] The token cache is being invalidate because project 95abb80de0e34a80a5332ba40aa8b2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:20.590690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.591312 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.592756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1357/2701] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:20 2022] DELETE /identity/v3/projects/95abb80de0e34a80a5332ba40aa8b2e2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:20.597383 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:20.607799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529e272-f634-46d5-ad6a-9eafc9985636 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:20.663760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:20.664387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:20.664671 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:20.664949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] PATH_INFO: `/v3/users/06b76970e8bd45d5909aa4fe9a86fe57` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:20.666116 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b76970e8bd45d5909aa4fe9a86fe57)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:20.668980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:20.714695 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b76970e8bd45d5909aa4fe9a86fe57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.715360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b76970e8bd45d5909aa4fe9a86fe57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.722786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b76970e8bd45d5909aa4fe9a86fe57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.730126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b76970e8bd45d5909aa4fe9a86fe57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.742776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b23f86e3-8178-41f2-8870-b006dccf18c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b76970e8bd45d5909aa4fe9a86fe57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:20.748766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1345/2702] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:20 2022] DELETE /identity/v3/users/06b76970e8bd45d5909aa4fe9a86fe57 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:21.034503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.035931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:21.049084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eac950-e0c6-4197-8ae5-ab4c340dfb6d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.105168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.105824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.106108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.106376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] PATH_INFO: `/v3/projects/6b3daae5417948858e9b9af256081d0a` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.107482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3daae5417948858e9b9af256081d0a)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.109360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.155225 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3daae5417948858e9b9af256081d0a', 'request_id': 'req-1646805b-2f4a-41b8-9174-b61a491a6e96'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:21.156911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:21.163660 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3daae5417948858e9b9af256081d0a', 'request_id': 'req-1646805b-2f4a-41b8-9174-b61a491a6e96'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:21.170491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3daae5417948858e9b9af256081d0a', 'request_id': 'req-1646805b-2f4a-41b8-9174-b61a491a6e96'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:21.180691 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] The token cache is being invalidate because project 6b3daae5417948858e9b9af256081d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:21.180977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:21.181595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1646805b-2f4a-41b8-9174-b61a491a6e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:21.183078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1358/2703] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:21 2022] DELETE /identity/v3/projects/6b3daae5417948858e9b9af256081d0a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:21.188553 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b75d1-2513-4d95-a895-5a95cd484df8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.188827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b75d1-2513-4d95-a895-5a95cd484df8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.189101 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b75d1-2513-4d95-a895-5a95cd484df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.208827 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-817b75d1-2513-4d95-a895-5a95cd484df8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:21.210075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-817b75d1-2513-4d95-a895-5a95cd484df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:21.248972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:21.271046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1346/2704] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:21.278750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ceb732-b766-4dec-9383-6f692d76c593 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.279032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ceb732-b766-4dec-9383-6f692d76c593 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.279303 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ceb732-b766-4dec-9383-6f692d76c593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.298365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-59ceb732-b766-4dec-9383-6f692d76c593 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:21.299544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-59ceb732-b766-4dec-9383-6f692d76c593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:21.307744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1359/2705] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:21.311910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b75d1-2513-4d95-a895-5a95cd484df8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.321997 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b75d1-2513-4d95-a895-5a95cd484df8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.326535 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.327129 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.327411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.327680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.328999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:33:21.329300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.330917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.342377 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1347/2706] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:33:21.346570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ceb732-b766-4dec-9383-6f692d76c593 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.355732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ceb732-b766-4dec-9383-6f692d76c593 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.360375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.360963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.361242 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.361514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.362174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.364054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.392392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1360/2707] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:21 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:21.396439 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.406237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e101cf2-dd72-4071-b911-0b1f6bcde99e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.460308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.460910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.461449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.461723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.462386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.464276 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.484991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1348/2708] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:21 2022] POST /identity/v3/users => generated 304 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:21.489214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.499545 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f269717-c974-48e8-a324-60455aff392a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.504949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.505551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.505868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.506144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.506787 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.508359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.518176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1361/2709] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:21.521950 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.531707 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e684d6b-152e-449a-9819-a253d08a7f91 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.536290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5bb24-de00-4a35-8d0b-bc459607b2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.536882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5bb24-de00-4a35-8d0b-bc459607b2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.537164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5bb24-de00-4a35-8d0b-bc459607b2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.537463 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5bb24-de00-4a35-8d0b-bc459607b2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.540375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5bb24-de00-4a35-8d0b-bc459607b2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.540375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5bb24-de00-4a35-8d0b-bc459607b2ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.551620 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1349/2710] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:21.555782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.566165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45105f91-cf9c-48f2-810c-c9374ba722ff admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.570712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.571356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.571632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.571902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] PATH_INFO: `/v3/projects/7b9a8ed80de346cab5889686e2904dc8/users/47363fa7ee334346a6a2d404d15b3e5e/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.586026 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9a8ed80de346cab5889686e2904dc8, user_id=47363fa7ee334346a6a2d404d15b3e5e, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.588870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.602365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1362/2711] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:21 2022] PUT /identity/v3/projects/7b9a8ed80de346cab5889686e2904dc8/users/47363fa7ee334346a6a2d404d15b3e5e/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:21.612760 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09639c8a-5f60-48e6-902b-920fbaf03540 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.613257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09639c8a-5f60-48e6-902b-920fbaf03540 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.613350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09639c8a-5f60-48e6-902b-920fbaf03540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.641256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-09639c8a-5f60-48e6-902b-920fbaf03540 None None] MFA Rules not processed for user `47363fa7ee334346a6a2d404d15b3e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:21.642496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09639c8a-5f60-48e6-902b-920fbaf03540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:21.703823 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1350/2712] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:21.708304 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.769627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abecaca-2c91-4f6c-9a49-1c1707b3686a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.774262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.774858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.775140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.775417 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.776038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.777940 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.807786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1363/2713] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:21 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:21.808903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09639c8a-5f60-48e6-902b-920fbaf03540 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.820595 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09639c8a-5f60-48e6-902b-920fbaf03540 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.896176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:21.917214 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.917907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.918190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.918460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.919106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.921000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.944307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1351/2714] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:21 2022] POST /identity/v3/users => generated 309 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:33:21.945413 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.955583 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018dcd5-0f04-42b0-a008-32ea6c7e35ee admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.960111 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.960702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.960979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.961506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:21.962174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:21.963737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:21.978430 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1364/2715] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:33:21.982731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:21.993147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1aaf6-a71b-4cb9-84a2-78753eae7f15 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:21.998236 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:21.998879 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:21.999158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:21.999429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] PATH_INFO: `/v3/projects/3f34d880bb7b488a98cb0317ce8f422a/users/9bea27d419264e6f97a8baf689ab39a3/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:22.013817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f34d880bb7b488a98cb0317ce8f422a, user_id=9bea27d419264e6f97a8baf689ab39a3, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:22.016471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:22.029197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1352/2716] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:21 2022] PUT /identity/v3/projects/3f34d880bb7b488a98cb0317ce8f422a/users/9bea27d419264e6f97a8baf689ab39a3/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:22.033528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:22.044273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb20d1d-6145-43bd-b87b-8c12bf511dee admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:22.089349 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:22.103778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a830ce-23fd-4bf6-a40f-009510a44acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:22.104384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a830ce-23fd-4bf6-a40f-009510a44acd admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:22.104664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a830ce-23fd-4bf6-a40f-009510a44acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:22.104939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a830ce-23fd-4bf6-a40f-009510a44acd admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:22.105584 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a830ce-23fd-4bf6-a40f-009510a44acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:22.107203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a830ce-23fd-4bf6-a40f-009510a44acd admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:22.117231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1365/2717] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:22.121544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:22.131822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb13b7-0077-4779-a8d2-36d736a9e362 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:22.136475 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:22.137160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:22.137442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:22.137716 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] PATH_INFO: `/v3/projects/3f34d880bb7b488a98cb0317ce8f422a/users/9bea27d419264e6f97a8baf689ab39a3/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:22.139713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f34d880bb7b488a98cb0317ce8f422a, user_id=9bea27d419264e6f97a8baf689ab39a3, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:22.142364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:22.152910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1353/2718] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:22 2022] PUT /identity/v3/projects/3f34d880bb7b488a98cb0317ce8f422a/users/9bea27d419264e6f97a8baf689ab39a3/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:22.165007 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7ee7-d0a3-42f1-8076-cd3eef27098b None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:22.165495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7ee7-d0a3-42f1-8076-cd3eef27098b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:22.165818 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7ee7-d0a3-42f1-8076-cd3eef27098b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:22.199024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0e1c7ee7-d0a3-42f1-8076-cd3eef27098b None None] MFA Rules not processed for user `9bea27d419264e6f97a8baf689ab39a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:22.200287 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0e1c7ee7-d0a3-42f1-8076-cd3eef27098b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:22.276046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1366/2719] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:33:22.288055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:22.289816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:22.350995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e311dc-9646-43a2-98df-739035c33226 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:22.354659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb28ba6-b767-4499-adcd-e76cc0c8e99e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:22.355263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28ba6-b767-4499-adcd-e76cc0c8e99e service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:22.355544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28ba6-b767-4499-adcd-e76cc0c8e99e service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:22.355817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28ba6-b767-4499-adcd-e76cc0c8e99e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:22.364625 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28ba6-b767-4499-adcd-e76cc0c8e99e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:22.366073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28ba6-b767-4499-adcd-e76cc0c8e99e service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:22.375296 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1354/2720] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:22.529343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c7ee7-d0a3-42f1-8076-cd3eef27098b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:22.539321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c7ee7-d0a3-42f1-8076-cd3eef27098b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:22.543197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:22.543801 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:22.544081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:22.544353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:22.605562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:22.607008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:22.615755 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1367/2721] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:24.283043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e89e6-d454-4514-b604-c958a1e03362 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:24.283501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e89e6-d454-4514-b604-c958a1e03362 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:24.283634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e89e6-d454-4514-b604-c958a1e03362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:24.302200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a48e89e6-d454-4514-b604-c958a1e03362 None None] MFA Rules not processed for user `f4fd2deca6124518ae1df9ed7977f350`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:24.303385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a48e89e6-d454-4514-b604-c958a1e03362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:24.358792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1355/2722] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:33:32.611679 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=69 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:34.199937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:34.261936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a894f19-1654-40e8-b563-dd43c3f695d4 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:34.266953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:34.267577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:34.267860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:34.268132 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] PATH_INFO: `/v3/users/47363fa7ee334346a6a2d404d15b3e5e` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:34.269252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47363fa7ee334346a6a2d404d15b3e5e)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:34.271185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:34.318532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47363fa7ee334346a6a2d404d15b3e5e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.327937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47363fa7ee334346a6a2d404d15b3e5e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.328785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47363fa7ee334346a6a2d404d15b3e5e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.337027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47363fa7ee334346a6a2d404d15b3e5e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.349746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47363fa7ee334346a6a2d404d15b3e5e'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.352202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:34.358564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1368/2723] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:34 2022] DELETE /identity/v3/users/47363fa7ee334346a6a2d404d15b3e5e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:34.363061 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:34.363350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:34.363626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:34.383937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:34.385313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:34.445814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1356/2724] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:34.456122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:34.457593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:34.515548 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddb495-f11e-494f-92b2-e1ce97994f1f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:34.519738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:34.520528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:34.520808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:34.521127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:34.531234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:34.532629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:34.542133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1369/2725] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:34.804969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:34.806416 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:34.819335 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3fee7-cffa-47d6-8259-7bc1db1dc45a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:34.824051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:34.824683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:34.824969 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:34.825249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] PATH_INFO: `/v3/projects/7b9a8ed80de346cab5889686e2904dc8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:34.826880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9a8ed80de346cab5889686e2904dc8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:34.829388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:34.875258 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9a8ed80de346cab5889686e2904dc8', 'request_id': 'req-3b608919-1887-46eb-80b5-ae8b849abd3a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.881208 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9a8ed80de346cab5889686e2904dc8', 'request_id': 'req-3b608919-1887-46eb-80b5-ae8b849abd3a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.886660 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9a8ed80de346cab5889686e2904dc8', 'request_id': 'req-3b608919-1887-46eb-80b5-ae8b849abd3a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.899089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] The token cache is being invalidate because project 7b9a8ed80de346cab5889686e2904dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:34.899374 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.900484 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:34.902228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1357/2726] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:34 2022] DELETE /identity/v3/projects/7b9a8ed80de346cab5889686e2904dc8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:33:34.906099 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:34.915423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7262f-d5e8-4c2c-9c64-73ab0e7242ef service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:34.933182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:34.972785 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:34.973408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:34.973684 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:34.973986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] PATH_INFO: `/v3/users/9bea27d419264e6f97a8baf689ab39a3` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:34.975092 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bea27d419264e6f97a8baf689ab39a3)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:34.976988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.024140 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bea27d419264e6f97a8baf689ab39a3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.031337 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bea27d419264e6f97a8baf689ab39a3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.031945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bea27d419264e6f97a8baf689ab39a3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.038727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bea27d419264e6f97a8baf689ab39a3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.051069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80a690a9-55c8-4598-9725-4459a0d5468e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bea27d419264e6f97a8baf689ab39a3'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.056884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1370/2727] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:34 2022] DELETE /identity/v3/users/9bea27d419264e6f97a8baf689ab39a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:35.338931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:35.340489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:35.355158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b608919-1887-46eb-80b5-ae8b849abd3a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:35.417850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:35.417850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.417850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.417850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] PATH_INFO: `/v3/projects/3f34d880bb7b488a98cb0317ce8f422a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.418211 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f34d880bb7b488a98cb0317ce8f422a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:35.419213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.452878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:35.471897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f34d880bb7b488a98cb0317ce8f422a', 'request_id': 'req-1555f6a6-97e2-4908-b993-b4a13a256ba8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.480597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f34d880bb7b488a98cb0317ce8f422a', 'request_id': 'req-1555f6a6-97e2-4908-b993-b4a13a256ba8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.488903 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f34d880bb7b488a98cb0317ce8f422a', 'request_id': 'req-1555f6a6-97e2-4908-b993-b4a13a256ba8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.502528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] The token cache is being invalidate because project 3f34d880bb7b488a98cb0317ce8f422a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:35.502819 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.503498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1555f6a6-97e2-4908-b993-b4a13a256ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:35.505003 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1358/2728] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:35 2022] DELETE /identity/v3/projects/3f34d880bb7b488a98cb0317ce8f422a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:35.509676 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37adead-f26d-4fa6-b4d9-48df9ef76a2d None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.510185 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37adead-f26d-4fa6-b4d9-48df9ef76a2d None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.510389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37adead-f26d-4fa6-b4d9-48df9ef76a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.529778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a37adead-f26d-4fa6-b4d9-48df9ef76a2d None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:35.531506 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a37adead-f26d-4fa6-b4d9-48df9ef76a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:35.592108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1371/2729] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:35.600627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4784cd3-ef12-4737-9665-f5fc083454b2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.600951 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4784cd3-ef12-4737-9665-f5fc083454b2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.601228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4784cd3-ef12-4737-9665-f5fc083454b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.623654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d4784cd3-ef12-4737-9665-f5fc083454b2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:35.624935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d4784cd3-ef12-4737-9665-f5fc083454b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:35.633043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1359/2730] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:35.637319 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37adead-f26d-4fa6-b4d9-48df9ef76a2d None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:35.648673 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37adead-f26d-4fa6-b4d9-48df9ef76a2d None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:35.653324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:35.653978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.654330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.654577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.655177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:33:35.655988 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:35.657550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.669681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1372/2731] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:33:35.674431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4784cd3-ef12-4737-9665-f5fc083454b2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:35.684162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4784cd3-ef12-4737-9665-f5fc083454b2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:35.689046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:35.689669 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.689978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.690257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.690912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:35.692818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.724126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1360/2732] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:35 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:35.728205 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:35.738435 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da974c12-7efb-4776-a075-1ae11a0ef001 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:35.766053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:35.800501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:35.801254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.801532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.801856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.802494 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:35.804353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.830415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1373/2733] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:35 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:35.831693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:35.842400 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec60b9-53dd-43c7-bfbe-318ee3a0800d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:35.847545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:35.848147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.848428 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.848699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.849407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:35.851037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.861138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1361/2734] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:33:35.865248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:35.875317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14ece3-e7b0-4ccf-b4bf-9372b5ea5a5a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:35.879871 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9da875-7568-4ce0-966c-6fcab5bae6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:35.880470 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da875-7568-4ce0-966c-6fcab5bae6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.880748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da875-7568-4ce0-966c-6fcab5bae6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.881024 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da875-7568-4ce0-966c-6fcab5bae6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.881672 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9da875-7568-4ce0-966c-6fcab5bae6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:35.883280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9da875-7568-4ce0-966c-6fcab5bae6d0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.893979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1374/2735] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:35.897986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:35.908348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d4982-40a5-4055-bbf5-c37729607617 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:35.913014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:35.913670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.913988 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.914266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] PATH_INFO: `/v3/projects/fd336505d33440879a3b994948af582a/users/c1aec5ec30a24730bde1d22aa02eb5dd/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.930526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd336505d33440879a3b994948af582a, user_id=c1aec5ec30a24730bde1d22aa02eb5dd, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:35.933624 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:35.947866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1362/2736] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:35 2022] PUT /identity/v3/projects/fd336505d33440879a3b994948af582a/users/c1aec5ec30a24730bde1d22aa02eb5dd/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:33:35.953528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c381d27-f19d-4b9a-98b8-e3a6d7180e30 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:35.953750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c381d27-f19d-4b9a-98b8-e3a6d7180e30 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:35.954164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c381d27-f19d-4b9a-98b8-e3a6d7180e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:35.974306 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8c381d27-f19d-4b9a-98b8-e3a6d7180e30 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:35.975528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8c381d27-f19d-4b9a-98b8-e3a6d7180e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:36.036356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1375/2737] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:36.046077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:36.047906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:36.106072 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb2040-24d4-44ba-ac57-4462d4cd406a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:36.109537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:36.110162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:36.110436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:36.110705 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:36.119391 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:36.120803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:36.130498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1363/2738] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:36.801071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c3ff4-6d74-4abd-98d3-a1e2f360aced None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:36.801696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c3ff4-6d74-4abd-98d3-a1e2f360aced None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:36.801696 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c3ff4-6d74-4abd-98d3-a1e2f360aced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:36.833245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d15c3ff4-6d74-4abd-98d3-a1e2f360aced None None] MFA Rules not processed for user `c1aec5ec30a24730bde1d22aa02eb5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:36.834566 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d15c3ff4-6d74-4abd-98d3-a1e2f360aced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:36.905000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1376/2739] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:36.910020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:36.922188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f004e-c734-451a-b091-8c9020456ced service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:36.926753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:36.927362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:36.927644 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:36.927916 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:36.928558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:36.930519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:36.960277 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1364/2740] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:36 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:33:36.962084 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c3ff4-6d74-4abd-98d3-a1e2f360aced None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:36.973240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c3ff4-6d74-4abd-98d3-a1e2f360aced None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:37.009606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:37.034258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:37.034860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:37.035198 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:37.035471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:37.036097 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:37.038037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:37.059552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1377/2741] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:36 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:33:37.063358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:37.073445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7410d6-39e5-44a2-86ea-810dbab8a9a4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:37.077995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:37.078594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:37.078900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:37.079301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:37.079944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:37.081568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:37.092289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1365/2742] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:37.096738 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:37.106609 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573345e-10bc-4af2-b988-a1b5c59c788b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:37.111839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:37.112477 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:37.112757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:37.113031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] PATH_INFO: `/v3/projects/9bb5832a380f405191f0800ac8564366/users/6079bf8164234bc680408274168ce1cc/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:37.127359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb5832a380f405191f0800ac8564366, user_id=6079bf8164234bc680408274168ce1cc, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:37.130017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:37.144987 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1378/2743] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:37 2022] PUT /identity/v3/projects/9bb5832a380f405191f0800ac8564366/users/6079bf8164234bc680408274168ce1cc/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:37.149295 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:37.159434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdd44a-1153-4852-9beb-1cf3278654c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:37.176304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:37.219022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:37.219626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:37.219910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:37.220185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:37.220822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:37.222454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:37.231976 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1366/2744] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:37.236207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:37.246141 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900532ac-14c9-4d06-8ddd-bc6489ad5872 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:37.250645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c3b43-153c-4191-bfcb-3f6449b62f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:37.251277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3b43-153c-4191-bfcb-3f6449b62f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:37.251553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3b43-153c-4191-bfcb-3f6449b62f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:37.251821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3b43-153c-4191-bfcb-3f6449b62f17 admin admin] PATH_INFO: `/v3/projects/9bb5832a380f405191f0800ac8564366/users/6079bf8164234bc680408274168ce1cc/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:37.253689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c3b43-153c-4191-bfcb-3f6449b62f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb5832a380f405191f0800ac8564366, user_id=6079bf8164234bc680408274168ce1cc, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:37.256299 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c3b43-153c-4191-bfcb-3f6449b62f17 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:37.266686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1379/2745] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:37 2022] PUT /identity/v3/projects/9bb5832a380f405191f0800ac8564366/users/6079bf8164234bc680408274168ce1cc/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:37.688954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:37.692567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:37.712746 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827b12-d37d-46d5-a1ab-4b47833ea389 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:37.785655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:37.786389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:37.786802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:37.787121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] PATH_INFO: `/v3/users/043a167d81eb473e90d757bdb497a3dd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:37.788352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=043a167d81eb473e90d757bdb497a3dd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:37.790375 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:38.006102 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043a167d81eb473e90d757bdb497a3dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.007981 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043a167d81eb473e90d757bdb497a3dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.018379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043a167d81eb473e90d757bdb497a3dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.032193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:38.032727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:38.033173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:38.035889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043a167d81eb473e90d757bdb497a3dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.054385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7bd8dcef-0e4a-4b2b-a5a0-e652925cd768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043a167d81eb473e90d757bdb497a3dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.064261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1367/2746] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:37 2022] DELETE /identity/v3/users/043a167d81eb473e90d757bdb497a3dd => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:38.071909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:38.072193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:38.072468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:38.093857 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] MFA Rules not processed for user `6079bf8164234bc680408274168ce1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:38.095330 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:38.106692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:38.107913 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:38.152616 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:38.187362 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1368/2747] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:33:38.197869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1380/2748] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:38.200066 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:38.201604 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:38.209495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:38.211463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:38.264561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd952c71-5a4d-4424-8cfd-d7235bee11dc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:38.268226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:38.268825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:38.269104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:38.269372 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:38.273931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914c62c-8dd5-4752-94c2-b4dadd881d32 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:38.277972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:38.278280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:38.278779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:38.279067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:38.279343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:38.279720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:38.281115 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:38.289235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1369/2749] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:38.311252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:38.344713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:38.346166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:38.355348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1381/2750] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:33:38.570692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:38.572270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:38.587304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07e1c-11ca-4b55-ba5f-a46f26b0ae47 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:38.592406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:38.593720 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:38.594112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:38.594557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] PATH_INFO: `/v3/projects/7ef33aff3d864a0eab321a82554e5d61` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:38.596462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef33aff3d864a0eab321a82554e5d61)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:38.599607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:38.654882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef33aff3d864a0eab321a82554e5d61', 'request_id': 'req-8a021673-617b-4d81-a28c-0bd0c651978d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.662690 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef33aff3d864a0eab321a82554e5d61', 'request_id': 'req-8a021673-617b-4d81-a28c-0bd0c651978d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.668962 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef33aff3d864a0eab321a82554e5d61', 'request_id': 'req-8a021673-617b-4d81-a28c-0bd0c651978d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.696162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] The token cache is being invalidate because project 7ef33aff3d864a0eab321a82554e5d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:38.696458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.697114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:38.699017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1370/2751] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:38 2022] DELETE /identity/v3/projects/7ef33aff3d864a0eab321a82554e5d61 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:39.747113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:39.748482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:39.760087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd74254-ca10-4d45-99cf-7d4d35fff015 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:39.807636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:39.808252 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:39.808507 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:39.808779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:39.810501 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:39.871513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:39.872912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:39.882258 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1382/2752] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:48.694549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:49.878085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=64 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:52.933044 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:52.934525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:52.998805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a021673-617b-4d81-a28c-0bd0c651978d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:53.003309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:53.003933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:53.004213 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:53.004483 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] PATH_INFO: `/v3/users/c1aec5ec30a24730bde1d22aa02eb5dd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:53.005577 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1aec5ec30a24730bde1d22aa02eb5dd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:53.007509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:53.052934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1aec5ec30a24730bde1d22aa02eb5dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.053566 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1aec5ec30a24730bde1d22aa02eb5dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.060465 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1aec5ec30a24730bde1d22aa02eb5dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.067654 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1aec5ec30a24730bde1d22aa02eb5dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.080192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1aec5ec30a24730bde1d22aa02eb5dd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.086331 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1371/2753] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:52 2022] DELETE /identity/v3/users/c1aec5ec30a24730bde1d22aa02eb5dd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:53.257686 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:53.268237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d96eb2-820f-4f2a-aca6-34bbd36bb846 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:53.325025 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:53.325644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:53.325954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:53.326226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] PATH_INFO: `/v3/projects/fd336505d33440879a3b994948af582a` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:53.327440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd336505d33440879a3b994948af582a)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:53.329888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:53.377978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd336505d33440879a3b994948af582a', 'request_id': 'req-68bf06d5-4c2e-4315-929b-554158f690e8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.384984 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd336505d33440879a3b994948af582a', 'request_id': 'req-68bf06d5-4c2e-4315-929b-554158f690e8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.391387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd336505d33440879a3b994948af582a', 'request_id': 'req-68bf06d5-4c2e-4315-929b-554158f690e8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.401295 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] The token cache is being invalidate because project fd336505d33440879a3b994948af582a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:53.401577 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.402216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.403861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1383/2754] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:53 2022] DELETE /identity/v3/projects/fd336505d33440879a3b994948af582a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:53.408139 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:53.409504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:53.421901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44446583-4dd0-47aa-a4f2-6f8d67f97347 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:53.476217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:53.476849 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:53.477125 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:53.477396 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] PATH_INFO: `/v3/users/6079bf8164234bc680408274168ce1cc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:53.478652 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6079bf8164234bc680408274168ce1cc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:53.480580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:53.501205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:53.527138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6079bf8164234bc680408274168ce1cc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.527771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6079bf8164234bc680408274168ce1cc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.535012 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6079bf8164234bc680408274168ce1cc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.542639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6079bf8164234bc680408274168ce1cc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.554473 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-189de1e2-826a-4901-8bdc-8fe5de060d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6079bf8164234bc680408274168ce1cc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.560171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1372/2755] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:33:53 2022] DELETE /identity/v3/users/6079bf8164234bc680408274168ce1cc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:53.732073 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:53.733611 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:53.747208 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf06d5-4c2e-4315-929b-554158f690e8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:53.803965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:53.804588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:53.804861 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:53.805128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] PATH_INFO: `/v3/projects/9bb5832a380f405191f0800ac8564366` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:53.806245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb5832a380f405191f0800ac8564366)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:53.808125 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:53.842144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:53.855122 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb5832a380f405191f0800ac8564366', 'request_id': 'req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.862121 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb5832a380f405191f0800ac8564366', 'request_id': 'req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.868083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb5832a380f405191f0800ac8564366', 'request_id': 'req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.878886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] The token cache is being invalidate because project 9bb5832a380f405191f0800ac8564366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:33:53.879167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.879788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-63e886ac-1f1a-4ba4-9fdf-2a3b56106b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:33:53.881229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1384/2756] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:33:53 2022] DELETE /identity/v3/projects/9bb5832a380f405191f0800ac8564366 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:53.886989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140cdec-5a32-4245-be5a-e6f0d134dab8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:53.887285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140cdec-5a32-4245-be5a-e6f0d134dab8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:53.887538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140cdec-5a32-4245-be5a-e6f0d134dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:53.906876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8140cdec-5a32-4245-be5a-e6f0d134dab8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:53.908048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8140cdec-5a32-4245-be5a-e6f0d134dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:53.966618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1373/2757] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:53.974998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c8dfe-554c-4c1a-888a-c67a991852e6 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:53.975288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c8dfe-554c-4c1a-888a-c67a991852e6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:53.975595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c8dfe-554c-4c1a-888a-c67a991852e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:53.995570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-7a0c8dfe-554c-4c1a-888a-c67a991852e6 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:53.996922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7a0c8dfe-554c-4c1a-888a-c67a991852e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:54.005436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1385/2758] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:54.009641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140cdec-5a32-4245-be5a-e6f0d134dab8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.019204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140cdec-5a32-4245-be5a-e6f0d134dab8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.023683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.024268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.024544 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.024811 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.025444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:33:54.025742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.027343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.038960 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1374/2759] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:33:54.044817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c8dfe-554c-4c1a-888a-c67a991852e6 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.054722 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c8dfe-554c-4c1a-888a-c67a991852e6 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.059359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.059951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.060232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.060631 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.061265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.063192 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.090129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1386/2760] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:54.094896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.104909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a7100-821b-4f73-8add-b03942f931bf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.160949 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.161549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.161871 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.162140 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.162753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.164627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.185321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1375/2761] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:33:54.189779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.199822 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e75b-e905-4aab-add2-8a1785e077f1 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.204883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.205476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.205753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.206053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.206690 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.208253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.219068 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1387/2762] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:33:54.222834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.232418 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b2bf9-6695-4a92-9d34-5b8fbf4c1c67 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.237037 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c087e-9492-4e23-86c7-368a9a68746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.237613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c087e-9492-4e23-86c7-368a9a68746a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.237929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c087e-9492-4e23-86c7-368a9a68746a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.238200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c087e-9492-4e23-86c7-368a9a68746a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.238825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c087e-9492-4e23-86c7-368a9a68746a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.240380 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c087e-9492-4e23-86c7-368a9a68746a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.250608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1376/2763] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:54.254633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.264600 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e1e78-27d6-48f3-9df5-adbe7b053e28 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.269796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.270448 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.270726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.271002 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] PATH_INFO: `/v3/projects/c9244559adcd46268d691d66c74fa18a/users/ec8ca19f1fe844609c323582403eff85/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.286085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9244559adcd46268d691d66c74fa18a, user_id=ec8ca19f1fe844609c323582403eff85, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.290562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.304667 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1388/2764] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:54 2022] PUT /identity/v3/projects/c9244559adcd46268d691d66c74fa18a/users/ec8ca19f1fe844609c323582403eff85/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:54.314674 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515ec8-e487-4dec-b35e-1dfe9acef323 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.315048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515ec8-e487-4dec-b35e-1dfe9acef323 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.315379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515ec8-e487-4dec-b35e-1dfe9acef323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.323833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:54.345736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-81515ec8-e487-4dec-b35e-1dfe9acef323 None None] MFA Rules not processed for user `ec8ca19f1fe844609c323582403eff85`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:54.346952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-81515ec8-e487-4dec-b35e-1dfe9acef323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:54.407628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1377/2765] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:54.412075 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.472196 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c34773-1820-4cc6-983d-adb6979d3030 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.476833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.477427 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.477962 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.478257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.478883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.480742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.506935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1389/2766] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:54.511166 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515ec8-e487-4dec-b35e-1dfe9acef323 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.523089 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515ec8-e487-4dec-b35e-1dfe9acef323 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.578578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.579239 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.579521 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.579788 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.580419 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.582343 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.603002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1378/2767] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:54.607137 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.617406 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54af153-7c6d-4042-ae05-e5ac0bb95c6a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.622248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.622848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.623130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.623410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.624085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.626183 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.636642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1390/2768] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:54.641100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.650704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c44896-2810-44b7-aacc-140e63254343 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.655325 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6768c-baf5-4c05-815b-91436e4fab5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.655922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6768c-baf5-4c05-815b-91436e4fab5a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.656200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6768c-baf5-4c05-815b-91436e4fab5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.656470 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6768c-baf5-4c05-815b-91436e4fab5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.657105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6768c-baf5-4c05-815b-91436e4fab5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.658710 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6768c-baf5-4c05-815b-91436e4fab5a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.668303 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1379/2769] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:54.672291 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.682193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16024435-1a95-490b-b364-f8758010d259 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.686724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.687868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.688143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.688410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] PATH_INFO: `/v3/projects/ad707204aa6a4d2b9b123bc371d03959/users/966d8ef756204632b8b7d2fcc7350916/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.702619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad707204aa6a4d2b9b123bc371d03959, user_id=966d8ef756204632b8b7d2fcc7350916, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.705751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.718247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1391/2770] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:54 2022] PUT /identity/v3/projects/ad707204aa6a4d2b9b123bc371d03959/users/966d8ef756204632b8b7d2fcc7350916/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:33:54.728574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216d281-ef59-42ec-bee9-42220270906a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.728803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216d281-ef59-42ec-bee9-42220270906a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.729078 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216d281-ef59-42ec-bee9-42220270906a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.756468 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b216d281-ef59-42ec-bee9-42220270906a None None] MFA Rules not processed for user `966d8ef756204632b8b7d2fcc7350916`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:54.757657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b216d281-ef59-42ec-bee9-42220270906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:54.819740 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1380/2771] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:54.824200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.833202 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:54.888936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2c238-02b5-4b28-b61e-9862147ad555 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.893901 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.894498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.894779 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.895054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.895724 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:54.897591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:54.923623 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1392/2772] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:54.927851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216d281-ef59-42ec-bee9-42220270906a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:54.933176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:54.940565 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216d281-ef59-42ec-bee9-42220270906a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:54.996006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:54.996745 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:54.996940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:54.997184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:54.997889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:55.000576 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:55.021405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1381/2773] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:33:54 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:33:55.025467 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:55.035687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8e673-89d3-4403-b93f-ec29bb486ed9 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:55.040309 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:55.040916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:55.041189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:55.041459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:55.042165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:55.043737 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:55.054524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1393/2774] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:55.059622 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:55.070051 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa486b2-6515-4707-986a-e435e9f01e0f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:55.074580 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:55.075223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:55.075504 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:55.075845 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] PATH_INFO: `/v3/projects/057c2c9150494fca898eef57ab46c779/users/99b889b215184a678a2ff5b4dc92ac68/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:55.090637 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] RBAC: Authorizing `identity:create_grant(project_id=057c2c9150494fca898eef57ab46c779, user_id=99b889b215184a678a2ff5b4dc92ac68, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:55.093285 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:55.106253 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1382/2775] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:55 2022] PUT /identity/v3/projects/057c2c9150494fca898eef57ab46c779/users/99b889b215184a678a2ff5b4dc92ac68/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:33:55.110621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:55.120314 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f44d78-1360-40c0-afbc-7c979396a5b6 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:55.177998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c1a1c-eb48-4c6c-8017-e5092cd188eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:55.178603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1a1c-eb48-4c6c-8017-e5092cd188eb admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:55.178883 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1a1c-eb48-4c6c-8017-e5092cd188eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:55.179150 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1a1c-eb48-4c6c-8017-e5092cd188eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:55.179780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c1a1c-eb48-4c6c-8017-e5092cd188eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:55.181343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c1a1c-eb48-4c6c-8017-e5092cd188eb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:55.190769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1394/2776] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:33:55.195019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:55.205127 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cdcc6-9969-4715-90d5-85efcd8d593a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:55.210385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:55.211298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:55.211586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:55.211862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] PATH_INFO: `/v3/projects/057c2c9150494fca898eef57ab46c779/users/99b889b215184a678a2ff5b4dc92ac68/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:55.213781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057c2c9150494fca898eef57ab46c779, user_id=99b889b215184a678a2ff5b4dc92ac68, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:55.216408 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:55.227195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1383/2777] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:33:55 2022] PUT /identity/v3/projects/057c2c9150494fca898eef57ab46c779/users/99b889b215184a678a2ff5b4dc92ac68/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:33:55.239190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:55.239591 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:55.239936 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:55.268028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] MFA Rules not processed for user `99b889b215184a678a2ff5b4dc92ac68`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:33:55.269231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:55.336489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1395/2778] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:33:55.348077 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:55.349793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:55.405293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc7f2d-8d15-4625-9d83-51a78f684317 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:55.409442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:55.410065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:55.410352 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:55.410630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:55.464168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:55.465567 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:55.474970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1384/2779] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:56.977839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:56.979239 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:33:56.985916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:56.992716 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef7062-f457-43ca-8855-ca14b088edf0 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:56.996529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:56.997167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:56.997446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:56.997904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:57.054588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:57.055932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:57.065207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1396/2780] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:33:57.577817 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:33:57.586684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6551f-8dcb-41d0-a9f0-a462c6c6175a service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:33:57.590284 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76801999-4a0e-4104-9b4d-02a74c5d6356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:33:57.590876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76801999-4a0e-4104-9b4d-02a74c5d6356 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:33:57.591162 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76801999-4a0e-4104-9b4d-02a74c5d6356 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:33:57.591434 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76801999-4a0e-4104-9b4d-02a74c5d6356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:33:57.598783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:33:57.601448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76801999-4a0e-4104-9b4d-02a74c5d6356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:33:57.603406 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76801999-4a0e-4104-9b4d-02a74c5d6356 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:33:57.614449 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1385/2781] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:33:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:34:01.477503 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:01.538175 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660c58d-52e0-4bc4-bda8-6ce2be88015c service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:01.542730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909025-c290-402d-9f4c-666639568388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:01.543348 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909025-c290-402d-9f4c-666639568388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:01.543624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909025-c290-402d-9f4c-666639568388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:01.543896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909025-c290-402d-9f4c-666639568388 admin admin] PATH_INFO: `/v3/users/ec8ca19f1fe844609c323582403eff85` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:01.545017 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19909025-c290-402d-9f4c-666639568388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8ca19f1fe844609c323582403eff85)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:01.546932 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19909025-c290-402d-9f4c-666639568388 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:01.593528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8ca19f1fe844609c323582403eff85'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:01.602277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8ca19f1fe844609c323582403eff85'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:01.602890 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8ca19f1fe844609c323582403eff85'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:01.609748 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8ca19f1fe844609c323582403eff85'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:01.621265 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8ca19f1fe844609c323582403eff85'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:01.627008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1397/2782] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:34:01 2022] DELETE /identity/v3/users/ec8ca19f1fe844609c323582403eff85 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:01.631574 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:01.632041 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:01.632132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:01.651773 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:01.652959 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:01.712665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1386/2783] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:01.722910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:01.724311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:01.747027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:01.794142 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909025-c290-402d-9f4c-666639568388 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:01.797743 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:01.798380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:01.798663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:01.798935 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:01.807704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:01.809129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:01.819310 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1398/2784] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:34:02.086423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:02.087828 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:02.101618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b414c-a4f7-4664-ab0a-c9104e4509ba None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:02.106628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:02.107318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:02.107599 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:02.107866 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] PATH_INFO: `/v3/projects/c9244559adcd46268d691d66c74fa18a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:02.109019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9244559adcd46268d691d66c74fa18a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:02.111081 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:02.159751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9244559adcd46268d691d66c74fa18a', 'request_id': 'req-cc1f1600-03f6-4363-8399-0dd9337cc8f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.165786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9244559adcd46268d691d66c74fa18a', 'request_id': 'req-cc1f1600-03f6-4363-8399-0dd9337cc8f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.171167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9244559adcd46268d691d66c74fa18a', 'request_id': 'req-cc1f1600-03f6-4363-8399-0dd9337cc8f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.186793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] The token cache is being invalidate because project c9244559adcd46268d691d66c74fa18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:34:02.187144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.187797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.189887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1387/2785] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:34:02 2022] DELETE /identity/v3/projects/c9244559adcd46268d691d66c74fa18a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:02.193824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:02.203193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e38b8-2441-42aa-8d23-86101688ad75 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:02.258581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:02.259117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:02.259394 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:02.259664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] PATH_INFO: `/v3/users/966d8ef756204632b8b7d2fcc7350916` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:02.260764 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=966d8ef756204632b8b7d2fcc7350916)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:02.262702 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:02.307906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966d8ef756204632b8b7d2fcc7350916'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.317833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966d8ef756204632b8b7d2fcc7350916'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.318443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966d8ef756204632b8b7d2fcc7350916'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.325361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966d8ef756204632b8b7d2fcc7350916'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.337562 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966d8ef756204632b8b7d2fcc7350916'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.343380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1399/2786] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:34:02 2022] DELETE /identity/v3/users/966d8ef756204632b8b7d2fcc7350916 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:02.611858 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:02.613401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:02.628407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f1600-03f6-4363-8399-0dd9337cc8f5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:02.685136 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:02.685814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:02.686104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:02.686379 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] PATH_INFO: `/v3/projects/ad707204aa6a4d2b9b123bc371d03959` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:02.687524 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad707204aa6a4d2b9b123bc371d03959)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:02.689410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:02.712025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:02.741776 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad707204aa6a4d2b9b123bc371d03959', 'request_id': 'req-8581c709-bd37-4662-936b-42cfbd6030ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.747780 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad707204aa6a4d2b9b123bc371d03959', 'request_id': 'req-8581c709-bd37-4662-936b-42cfbd6030ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.753188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad707204aa6a4d2b9b123bc371d03959', 'request_id': 'req-8581c709-bd37-4662-936b-42cfbd6030ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.765084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] The token cache is being invalidate because project ad707204aa6a4d2b9b123bc371d03959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:34:02.765310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.765963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.767403 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1388/2787] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:34:02 2022] DELETE /identity/v3/projects/ad707204aa6a4d2b9b123bc371d03959 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:02.771403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:02.772909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:02.785927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9bac9-7465-4dba-b1cf-06d0274fa952 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:02.815963 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:02.846046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:02.846669 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:02.846951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:02.847223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] PATH_INFO: `/v3/users/99b889b215184a678a2ff5b4dc92ac68` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:02.848403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b889b215184a678a2ff5b4dc92ac68)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:02.850338 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:02.896478 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b889b215184a678a2ff5b4dc92ac68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.903651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b889b215184a678a2ff5b4dc92ac68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.904298 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b889b215184a678a2ff5b4dc92ac68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.911771 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b889b215184a678a2ff5b4dc92ac68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.924001 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-0ebc4418-9a68-4353-82ca-b0d82eb27b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b889b215184a678a2ff5b4dc92ac68'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:02.929792 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1400/2788] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:34:02 2022] DELETE /identity/v3/users/99b889b215184a678a2ff5b4dc92ac68 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:03.212512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.213991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:03.227561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581c709-bd37-4662-936b-42cfbd6030ab admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.283697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.284348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.284684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.284900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] PATH_INFO: `/v3/projects/057c2c9150494fca898eef57ab46c779` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.286112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=057c2c9150494fca898eef57ab46c779)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.287996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.335683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057c2c9150494fca898eef57ab46c779', 'request_id': 'req-38d74ab4-8672-4150-9bb1-c479d425fd4a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:03.342023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057c2c9150494fca898eef57ab46c779', 'request_id': 'req-38d74ab4-8672-4150-9bb1-c479d425fd4a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:03.348725 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057c2c9150494fca898eef57ab46c779', 'request_id': 'req-38d74ab4-8672-4150-9bb1-c479d425fd4a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:03.362297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] The token cache is being invalidate because project 057c2c9150494fca898eef57ab46c779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:34:03.362592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:03.363223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-38d74ab4-8672-4150-9bb1-c479d425fd4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:34:03.364699 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1389/2789] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:34:03 2022] DELETE /identity/v3/projects/057c2c9150494fca898eef57ab46c779 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:03.369499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d065e-304e-4fe6-a09d-d88c0a377430 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.369813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d065e-304e-4fe6-a09d-d88c0a377430 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.370093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d065e-304e-4fe6-a09d-d88c0a377430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.390283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ef2d065e-304e-4fe6-a09d-d88c0a377430 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:03.391523 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ef2d065e-304e-4fe6-a09d-d88c0a377430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:03.448912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1401/2790] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:03.457251 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7df1d6-799c-4112-9787-39b632b0ebd8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.457559 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7df1d6-799c-4112-9787-39b632b0ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.457863 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7df1d6-799c-4112-9787-39b632b0ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.477084 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8a7df1d6-799c-4112-9787-39b632b0ebd8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:03.478328 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8a7df1d6-799c-4112-9787-39b632b0ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:03.485996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1390/2791] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:03.490087 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d065e-304e-4fe6-a09d-d88c0a377430 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.499481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d065e-304e-4fe6-a09d-d88c0a377430 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.504579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.505163 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.505441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.505713 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.506387 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:34:03.506692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.508256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.520162 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1402/2792] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:34:03.524120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7df1d6-799c-4112-9787-39b632b0ebd8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.533014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7df1d6-799c-4112-9787-39b632b0ebd8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.537680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.538247 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.538550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.538820 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.539436 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.541297 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.568309 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1391/2793] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:34:03 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:34:03.572382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.583056 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b24919-bdaa-49fd-b575-59f6c832c057 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.615944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:03.641032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.641568 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.641899 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.642807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.643431 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.645316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.666055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1403/2794] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:34:03 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:34:03.670171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.680405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3b70b-ad85-47bc-859f-2a1687cd2bc2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.684946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.685538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.686126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.686126 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.686738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.688389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.694545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:03.700160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1392/2795] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:34:03.703998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.714811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78560ff9-6749-4316-8f2b-df5db897b96a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.719381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e9941-df0f-4d29-bb17-872bfe6df255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.719976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e9941-df0f-4d29-bb17-872bfe6df255 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.720257 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e9941-df0f-4d29-bb17-872bfe6df255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.720530 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e9941-df0f-4d29-bb17-872bfe6df255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.721170 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e9941-df0f-4d29-bb17-872bfe6df255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.722780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e9941-df0f-4d29-bb17-872bfe6df255 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.732415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1404/2796] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:34:03.736822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.746367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292e8cd-e6b2-4e53-96e9-9eca285cbbba admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.750861 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.752502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.752783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.753050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] PATH_INFO: `/v3/projects/0bfa97b89b0b4bdba0b64d6c2d1a58f9/users/6e2db3de0da54265adb783c13c377704/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.767036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfa97b89b0b4bdba0b64d6c2d1a58f9, user_id=6e2db3de0da54265adb783c13c377704, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.769633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.781696 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1393/2797] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:34:03 2022] PUT /identity/v3/projects/0bfa97b89b0b4bdba0b64d6c2d1a58f9/users/6e2db3de0da54265adb783c13c377704/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:03.787033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5063ef-260e-4dff-a207-dbd25f22f741 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.787538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5063ef-260e-4dff-a207-dbd25f22f741 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.787629 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5063ef-260e-4dff-a207-dbd25f22f741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.806638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-9c5063ef-260e-4dff-a207-dbd25f22f741 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:03.807746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-9c5063ef-260e-4dff-a207-dbd25f22f741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:03.864200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1405/2798] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:03.873870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:03.875555 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:03.931272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3050e-0202-4e81-b5e5-830972af0782 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:03.934867 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:03.935462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:03.935744 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:03.936019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:03.945528 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:03.946964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:03.956273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1394/2799] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:34:07.882143 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70763af-8641-4424-8c5b-f02a4af9a964 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:07.882674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70763af-8641-4424-8c5b-f02a4af9a964 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:07.882674 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70763af-8641-4424-8c5b-f02a4af9a964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:07.912264 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e70763af-8641-4424-8c5b-f02a4af9a964 None None] MFA Rules not processed for user `6e2db3de0da54265adb783c13c377704`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:07.913553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e70763af-8641-4424-8c5b-f02a4af9a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:07.985359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1406/2800] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:07.990438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:08.000606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e440894-0f7d-4156-8a7c-bea22aba09cd service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:08.006330 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:08.007164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:08.007458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:08.007803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:08.008427 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:08.010635 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:08.038541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1395/2801] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:34:07 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:34:08.042757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70763af-8641-4424-8c5b-f02a4af9a964 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:08.054182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70763af-8641-4424-8c5b-f02a4af9a964 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:08.107972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:08.125876 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:08.126571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:08.126858 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:08.127135 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:08.127898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:08.129837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:08.153811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1407/2802] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:34:08 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:34:08.158034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:08.168627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297667-ef70-4946-a88d-62f819819dbb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:08.173743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:08.174451 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:08.174719 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:08.175005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:08.175607 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:08.177197 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:08.189283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1396/2803] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:34:08.193190 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:08.203607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc8834-2639-4862-ba3c-eab30422a979 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:08.208549 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:08.209210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:08.209489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:08.209795 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] PATH_INFO: `/v3/projects/16f112f1c1f44679a19f0b6c2a0f793f/users/0da7e10395324ddea9f73d2f09e87730/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:08.229134 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f112f1c1f44679a19f0b6c2a0f793f, user_id=0da7e10395324ddea9f73d2f09e87730, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:08.232060 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:08.246754 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1408/2804] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:34:08 2022] PUT /identity/v3/projects/16f112f1c1f44679a19f0b6c2a0f793f/users/0da7e10395324ddea9f73d2f09e87730/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:34:08.252354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:08.262458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc0c22-d5be-47c8-859a-6f2b23daf972 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:08.321689 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d8276-712b-4cce-9c83-5b0b170ebac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:08.322324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d8276-712b-4cce-9c83-5b0b170ebac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:08.322597 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d8276-712b-4cce-9c83-5b0b170ebac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:08.322864 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d8276-712b-4cce-9c83-5b0b170ebac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:08.323498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d8276-712b-4cce-9c83-5b0b170ebac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:08.325064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d8276-712b-4cce-9c83-5b0b170ebac8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:08.336928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1397/2805] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:34:08.340943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:08.351608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57c55c-7b0d-4ad3-88e2-b6ddfcbc6e5e admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:08.356519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:08.357168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:08.357444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:08.357711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] PATH_INFO: `/v3/projects/16f112f1c1f44679a19f0b6c2a0f793f/users/0da7e10395324ddea9f73d2f09e87730/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:08.359638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f112f1c1f44679a19f0b6c2a0f793f, user_id=0da7e10395324ddea9f73d2f09e87730, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:08.362323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:08.373105 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1409/2806] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:34:08 2022] PUT /identity/v3/projects/16f112f1c1f44679a19f0b6c2a0f793f/users/0da7e10395324ddea9f73d2f09e87730/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:34:12.700596 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a53a-22e2-402d-b536-1631c4512326 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:12.701147 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a53a-22e2-402d-b536-1631c4512326 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:12.701792 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a53a-22e2-402d-b536-1631c4512326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:12.749533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2336a53a-22e2-402d-b536-1631c4512326 None None] MFA Rules not processed for user `0da7e10395324ddea9f73d2f09e87730`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:12.751002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2336a53a-22e2-402d-b536-1631c4512326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:12.795241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:12.840444 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1398/2807] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:12.853098 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:12.854894 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:12.939992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e1129-dc61-4c36-b4f7-a81a1e516fc4 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:12.943659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a24bc4a0c4f480192d87a27a243acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:12.944262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:12.944541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:12.944811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:13.008283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:13.009840 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:13.019854 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1410/2808] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:34:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:34:13.168317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:13.168603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:13.169071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:13.189939 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:13.191150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:13.251548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1399/2809] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:13.298719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:13.300200 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:13.335108 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:13.368760 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e31836-9d66-404c-b095-7e457e2c514c service glance] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:13.373704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:13.374440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:13.374720 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:13.374994 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:13.383510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:13.385061 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:13.393451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1411/2810] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:34:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:34:14.576256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:14.581256 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:14.702623 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34e05b-45e0-4bc6-809e-b697f055ce9d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:14.716888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c861c26-de0e-49f0-9e36-d438f4caabd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:14.717742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c861c26-de0e-49f0-9e36-d438f4caabd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:14.717742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c861c26-de0e-49f0-9e36-d438f4caabd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:14.717742 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c861c26-de0e-49f0-9e36-d438f4caabd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:14.719878 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c861c26-de0e-49f0-9e36-d438f4caabd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:14.732629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c861c26-de0e-49f0-9e36-d438f4caabd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:14.734768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c861c26-de0e-49f0-9e36-d438f4caabd2 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:14.744498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1400/2811] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:34:16.522637 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:16.525062 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:16.603164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03179d-2653-4522-8702-b7c46a28d882 service swift] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:16.607815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917abc59-5834-4d39-a14a-39399b75d677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:16.608742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917abc59-5834-4d39-a14a-39399b75d677 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:16.609244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917abc59-5834-4d39-a14a-39399b75d677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:16.609784 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917abc59-5834-4d39-a14a-39399b75d677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:16.617460 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-917abc59-5834-4d39-a14a-39399b75d677 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:16.733741 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917abc59-5834-4d39-a14a-39399b75d677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:16.738377 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917abc59-5834-4d39-a14a-39399b75d677 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:16.773315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1412/2812] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:34:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:34:23.040399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a618e-3ac9-4148-9c78-8ce7e43e5844 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:23.040964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a618e-3ac9-4148-9c78-8ce7e43e5844 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:23.040964 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a618e-3ac9-4148-9c78-8ce7e43e5844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:23.060165 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7d0a618e-3ac9-4148-9c78-8ce7e43e5844 None None] MFA Rules not processed for user `6e2db3de0da54265adb783c13c377704`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:23.061369 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7d0a618e-3ac9-4148-9c78-8ce7e43e5844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:23.069836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1401/2813] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:25.279408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630939fb-1d0f-42b5-add2-b927edfa0289 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:25.279865 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630939fb-1d0f-42b5-add2-b927edfa0289 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:25.279964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630939fb-1d0f-42b5-add2-b927edfa0289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:25.303735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-630939fb-1d0f-42b5-add2-b927edfa0289 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:25.304977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-630939fb-1d0f-42b5-add2-b927edfa0289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:25.313347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1413/2814] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Sep 19 12:34:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:25.327785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a618e-3ac9-4148-9c78-8ce7e43e5844 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:25.340282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a618e-3ac9-4148-9c78-8ce7e43e5844 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:25.345691 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:25.346317 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:25.346592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:25.346881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:25.355300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:25.356853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:25.370270 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1402/2815] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:34:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:34:35.297788 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=74 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:35.365676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=75 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:39.185297 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ee0f-ebc6-48d9-afb1-7d17c77a5bf0 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:39.185856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ee0f-ebc6-48d9-afb1-7d17c77a5bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:39.185856 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ee0f-ebc6-48d9-afb1-7d17c77a5bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:39.205916 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-cfe5ee0f-ebc6-48d9-afb1-7d17c77a5bf0 None None] MFA Rules not processed for user `f4fd2deca6124518ae1df9ed7977f350`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:34:39.207153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-cfe5ee0f-ebc6-48d9-afb1-7d17c77a5bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:34:39.268315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1414/2816] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:34:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:34:49.260892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=25 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:34:54.365308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:34:54.375222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed18231-4317-41c5-b231-869d4eab107c service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:34:54.379299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:34:54.379898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:34:54.380178 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:34:54.380445 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:34:54.392459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:34:54.393929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:34:54.403546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1403/2817] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:34:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:35:04.399412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:35:42.901746 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a721d-413f-4be7-9b27-b50f355000a7 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:35:42.902321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a721d-413f-4be7-9b27-b50f355000a7 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:35:42.902321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a721d-413f-4be7-9b27-b50f355000a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:35:42.921375 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-be9a721d-413f-4be7-9b27-b50f355000a7 None None] MFA Rules not processed for user `f4fd2deca6124518ae1df9ed7977f350`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:35:42.922710 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-be9a721d-413f-4be7-9b27-b50f355000a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:35:42.929595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1415/2818] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:35:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:35:52.916353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:00.910582 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:00.912241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:01.025073 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f52e3-1436-4dd8-afa5-4c79cd2d8d3b service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:01.031306 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:01.032224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:01.032364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:01.032594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:01.034386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:01.047433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:01.049167 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:01.060970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1404/2819] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:11.056134 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=47 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:13.974992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867742b-5af5-4af4-94aa-03c6be353d80 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:13.975463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867742b-5af5-4af4-94aa-03c6be353d80 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:13.975608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867742b-5af5-4af4-94aa-03c6be353d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:13.994315 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-0867742b-5af5-4af4-94aa-03c6be353d80 None None] MFA Rules not processed for user `79ac4874ca5043ac9ef8a47f999cac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:13.995495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-0867742b-5af5-4af4-94aa-03c6be353d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:14.002476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1416/2820] 10.222.0.88 () {58 vars in 967 bytes} [Mon Sep 19 12:36:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:14.043703 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:14.055138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd0fad-2a5e-4fa7-bba7-b43318661607 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:14.060060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1924693bdf934b15b1a784abfefdea5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:14.060653 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:14.060932 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:14.061204 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:14.070028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:14.071659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:14.080319 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1405/2821] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Sep 19 12:36:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:36:21.150409 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867742b-5af5-4af4-94aa-03c6be353d80 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:21.214905 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867742b-5af5-4af4-94aa-03c6be353d80 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:21.219544 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:21.220173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:21.220456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:21.220734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] PATH_INFO: `/v3/users/6e2db3de0da54265adb783c13c377704` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:21.221931 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2db3de0da54265adb783c13c377704)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:21.223863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:21.269615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2db3de0da54265adb783c13c377704'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.276404 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2db3de0da54265adb783c13c377704'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.277020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2db3de0da54265adb783c13c377704'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.283992 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2db3de0da54265adb783c13c377704'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.295643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2db3de0da54265adb783c13c377704'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.301633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1417/2822] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:36:21 2022] DELETE /identity/v3/users/6e2db3de0da54265adb783c13c377704 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:21.475783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:21.486443 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8fdbe-540c-4db2-b132-45827745c9ff service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:21.542280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:21.542905 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:21.543186 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:21.543461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] PATH_INFO: `/v3/projects/0bfa97b89b0b4bdba0b64d6c2d1a58f9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:21.544564 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bfa97b89b0b4bdba0b64d6c2d1a58f9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:21.546491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:21.593067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bfa97b89b0b4bdba0b64d6c2d1a58f9', 'request_id': 'req-bbff2455-6385-4e10-b5d4-fdd8043131ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.599603 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bfa97b89b0b4bdba0b64d6c2d1a58f9', 'request_id': 'req-bbff2455-6385-4e10-b5d4-fdd8043131ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.604991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bfa97b89b0b4bdba0b64d6c2d1a58f9', 'request_id': 'req-bbff2455-6385-4e10-b5d4-fdd8043131ad'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.616874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] The token cache is being invalidate because project 0bfa97b89b0b4bdba0b64d6c2d1a58f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:36:21.617150 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.617777 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.619237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1406/2823] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:36:21 2022] DELETE /identity/v3/projects/0bfa97b89b0b4bdba0b64d6c2d1a58f9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:21.623248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:21.624644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:21.638107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a9d02-9896-4ab5-8d32-4ce7fc6b36ef admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:21.695459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:21.696086 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:21.696363 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:21.696634 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] PATH_INFO: `/v3/users/0da7e10395324ddea9f73d2f09e87730` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:21.697747 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da7e10395324ddea9f73d2f09e87730)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:21.699658 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:21.713514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:21.744513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da7e10395324ddea9f73d2f09e87730'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.750954 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da7e10395324ddea9f73d2f09e87730'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.751749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da7e10395324ddea9f73d2f09e87730'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.759130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da7e10395324ddea9f73d2f09e87730'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.771380 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-08f9afa9-057f-4e36-8fc8-7d86a8db11eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da7e10395324ddea9f73d2f09e87730'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:21.777078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1418/2824] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:36:21 2022] DELETE /identity/v3/users/0da7e10395324ddea9f73d2f09e87730 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:21.948183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:21.949670 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:21.964865 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2455-6385-4e10-b5d4-fdd8043131ad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.022627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.023255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.023542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.023815 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] PATH_INFO: `/v3/projects/16f112f1c1f44679a19f0b6c2a0f793f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.024917 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f112f1c1f44679a19f0b6c2a0f793f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.026852 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.046775 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:22.073605 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f112f1c1f44679a19f0b6c2a0f793f', 'request_id': 'req-a219019a-fa5a-4b41-9360-08a026d1e387'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:22.081156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f112f1c1f44679a19f0b6c2a0f793f', 'request_id': 'req-a219019a-fa5a-4b41-9360-08a026d1e387'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:22.088794 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f112f1c1f44679a19f0b6c2a0f793f', 'request_id': 'req-a219019a-fa5a-4b41-9360-08a026d1e387'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:22.102004 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] The token cache is being invalidate because project 16f112f1c1f44679a19f0b6c2a0f793f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:36:22.102291 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:22.102931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a219019a-fa5a-4b41-9360-08a026d1e387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:22.104440 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1407/2825] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:36:21 2022] DELETE /identity/v3/projects/16f112f1c1f44679a19f0b6c2a0f793f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:22.110173 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387dd55-6f27-40b0-9de0-2e1a81a2be6f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.110497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387dd55-6f27-40b0-9de0-2e1a81a2be6f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.110812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387dd55-6f27-40b0-9de0-2e1a81a2be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.168833 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e387dd55-6f27-40b0-9de0-2e1a81a2be6f None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:22.170124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e387dd55-6f27-40b0-9de0-2e1a81a2be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:22.231057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1419/2826] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:22.239025 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98462c67-5617-41c2-b6ec-afebf560d7ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.239816 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98462c67-5617-41c2-b6ec-afebf560d7ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.240067 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98462c67-5617-41c2-b6ec-afebf560d7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.260054 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-98462c67-5617-41c2-b6ec-afebf560d7ee None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:22.261238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-98462c67-5617-41c2-b6ec-afebf560d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:22.269121 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1408/2827] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:22.273169 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387dd55-6f27-40b0-9de0-2e1a81a2be6f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:22.283318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387dd55-6f27-40b0-9de0-2e1a81a2be6f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.288389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.288986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.289290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.289586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.290277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:36:22.290581 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.292156 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.304155 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1420/2828] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:36:22.308342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98462c67-5617-41c2-b6ec-afebf560d7ee None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:22.317327 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98462c67-5617-41c2-b6ec-afebf560d7ee None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.321882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.322461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.322735 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.323000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.323612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.325454 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.361881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1409/2829] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:36:22 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:36:22.366575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:22.376389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0927c9-3f69-45f9-987e-01ee02e5554c admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.430498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:22.443378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.443981 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.444261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.444533 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.445174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.447106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.468410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1421/2830] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:36:22 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:36:22.472684 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:22.483308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896ddc6-16d4-40f8-bd4f-5e34e47d2ac8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.487857 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.488452 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.488731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.489002 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.490324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.491896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.501550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1410/2831] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:36:22.505350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:22.515096 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1289-b9bd-4c59-acec-9e603c0b8aff admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.519639 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d97a9e-adc4-46b7-98ea-286a298477e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.520224 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97a9e-adc4-46b7-98ea-286a298477e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.520532 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97a9e-adc4-46b7-98ea-286a298477e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.520805 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97a9e-adc4-46b7-98ea-286a298477e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.521442 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d97a9e-adc4-46b7-98ea-286a298477e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.523069 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d97a9e-adc4-46b7-98ea-286a298477e9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.533403 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1422/2832] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:36:22.537875 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:22.547442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92904ca-f200-4652-a89a-a95bc0afa972 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.552507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.553132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.553407 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.553678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] PATH_INFO: `/v3/projects/84a8601fd7bf4667a9b6901f6b4147cf/users/49c6e6d0e00449eaabc76b1211ec8c43/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.567568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a8601fd7bf4667a9b6901f6b4147cf, user_id=49c6e6d0e00449eaabc76b1211ec8c43, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.570200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.582279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1411/2833] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:36:22 2022] PUT /identity/v3/projects/84a8601fd7bf4667a9b6901f6b4147cf/users/49c6e6d0e00449eaabc76b1211ec8c43/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:36:22.587397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d4190-2404-430b-9bed-126610a18eb9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.587685 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d4190-2404-430b-9bed-126610a18eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.587964 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d4190-2404-430b-9bed-126610a18eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.607308 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-ab1d4190-2404-430b-9bed-126610a18eb9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:22.608556 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ab1d4190-2404-430b-9bed-126610a18eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:22.666624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1423/2834] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:36:22.676493 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:22.678933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:22.740731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c55e79-0a34-4035-8ecc-7b92861f10f5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:22.744665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:22.745278 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:22.745556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:22.745851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:22.755053 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:22.756471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:22.765888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1412/2835] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:26.698606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b17a04-fb54-4f58-8570-49f5c4989adf None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:26.699124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b17a04-fb54-4f58-8570-49f5c4989adf None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:26.699124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b17a04-fb54-4f58-8570-49f5c4989adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:26.732551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e8b17a04-fb54-4f58-8570-49f5c4989adf None None] MFA Rules not processed for user `49c6e6d0e00449eaabc76b1211ec8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:26.733974 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e8b17a04-fb54-4f58-8570-49f5c4989adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:26.833920 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1424/2836] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:26.838509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:26.873117 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16879b2-40de-4038-b0fb-887d886fead6 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:26.878300 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:26.878910 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:26.879191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:26.879467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:26.880643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:26.882614 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:26.898189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:26.916545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1413/2837] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:36:26 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:36:26.920572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b17a04-fb54-4f58-8570-49f5c4989adf None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:26.931594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b17a04-fb54-4f58-8570-49f5c4989adf None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:26.972990 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:26.993116 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:26.993643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:26.993957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:26.994234 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:26.994904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:26.996808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:27.021521 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1425/2838] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:36:26 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:36:27.025793 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:27.039160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835767ed-caeb-439f-83e6-6ddf8185b844 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:27.043944 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:27.044569 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:27.044850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:27.045123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:27.045790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:27.047392 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:27.057963 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1414/2839] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:36:27.062139 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:27.073035 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710018-4e5d-416d-b400-874983a390c5 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:27.078949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:27.079734 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:27.080021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:27.080300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] PATH_INFO: `/v3/projects/d342593fce454ca6a9d9890916191dcb/users/ea7915e975f946b0b8d854b02b96da77/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:27.096727 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d342593fce454ca6a9d9890916191dcb, user_id=ea7915e975f946b0b8d854b02b96da77, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:27.099378 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:27.112925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1426/2840] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:36:27 2022] PUT /identity/v3/projects/d342593fce454ca6a9d9890916191dcb/users/ea7915e975f946b0b8d854b02b96da77/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:27.116876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:27.127188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1825cbf-76a5-4673-88e6-fd8a958e64ac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:27.187423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8ed0-787e-42f2-9e5d-1ca10629ae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:27.188036 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8ed0-787e-42f2-9e5d-1ca10629ae6d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:27.188313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8ed0-787e-42f2-9e5d-1ca10629ae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:27.188584 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8ed0-787e-42f2-9e5d-1ca10629ae6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:27.189225 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f8ed0-787e-42f2-9e5d-1ca10629ae6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:27.190862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f8ed0-787e-42f2-9e5d-1ca10629ae6d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:27.200992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1415/2841] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:36:27.205120 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:27.215451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181ffc3-8982-4161-b3da-7630ba38f9e8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:27.219991 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:27.220633 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:27.220912 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:27.221184 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] PATH_INFO: `/v3/projects/d342593fce454ca6a9d9890916191dcb/users/ea7915e975f946b0b8d854b02b96da77/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:27.223762 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d342593fce454ca6a9d9890916191dcb, user_id=ea7915e975f946b0b8d854b02b96da77, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:27.226567 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:27.236969 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1427/2842] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:36:27 2022] PUT /identity/v3/projects/d342593fce454ca6a9d9890916191dcb/users/ea7915e975f946b0b8d854b02b96da77/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:31.662884 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93c15e-74f5-4149-8ced-03d17ec84a77 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:31.663342 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93c15e-74f5-4149-8ced-03d17ec84a77 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:31.663461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93c15e-74f5-4149-8ced-03d17ec84a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:31.701027 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dc93c15e-74f5-4149-8ced-03d17ec84a77 None None] MFA Rules not processed for user `ea7915e975f946b0b8d854b02b96da77`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:31.702272 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc93c15e-74f5-4149-8ced-03d17ec84a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:31.807351 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1416/2843] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:31.823835 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:31.825604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:31.944486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e7ddc-8a46-41c0-bfcd-db7d9068dea8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:31.950522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:31.951249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:31.951624 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:31.952009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:32.053094 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:32.055009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:32.073070 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1428/2844] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:33.221111 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:33.221628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:33.221883 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:33.243290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] MFA Rules not processed for user `49c6e6d0e00449eaabc76b1211ec8c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:33.244730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:33.251522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1417/2845] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:41.177392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:41.178878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:41.211437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:41.239535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718fa45-e22e-42b8-827e-c95795c8e5c1 service nova] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:41.243813 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:41.244414 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:41.244694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:41.244965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:41.246721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:41.259753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:41.261158 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:41.271975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1429/2846] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:43.237099 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:45.095043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:45.096799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:45.117462 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3f9ba-31ef-402f-bcba-d6455642e8be None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:45.124095 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:45.125035 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:45.125572 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:45.126478 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:45.128925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:45.149158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:45.152542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:45.170205 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1418/2847] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:51.267194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=13 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:53.945496 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:53.947177 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:54.016397 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a36de-0b10-4fc2-a8c1-2cd1038ea1b5 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.021203 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.021897 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.022179 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.022468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] PATH_INFO: `/v3/users/f4fd2deca6124518ae1df9ed7977f350` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.023571 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4fd2deca6124518ae1df9ed7977f350)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.025505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.073382 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4fd2deca6124518ae1df9ed7977f350'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.080641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4fd2deca6124518ae1df9ed7977f350'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.081278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4fd2deca6124518ae1df9ed7977f350'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.088838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4fd2deca6124518ae1df9ed7977f350'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.100313 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3e106a6f-2b75-46b8-b184-54c4038b7a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4fd2deca6124518ae1df9ed7977f350'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.106659 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1430/2848] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:36:53 2022] DELETE /identity/v3/users/f4fd2deca6124518ae1df9ed7977f350 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:54.276241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.296087 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab9c73-8abe-4a62-ad2b-0fe42801206f service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.360812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.361450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.361730 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.363812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] PATH_INFO: `/v3/projects/3da2a386a837446dad4bf8b8506e418d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.364958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3da2a386a837446dad4bf8b8506e418d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.366900 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.414876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3da2a386a837446dad4bf8b8506e418d', 'request_id': 'req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.420906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3da2a386a837446dad4bf8b8506e418d', 'request_id': 'req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.426329 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3da2a386a837446dad4bf8b8506e418d', 'request_id': 'req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.442161 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] The token cache is being invalidate because project 3da2a386a837446dad4bf8b8506e418d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:36:54.442453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.443114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-89f514e1-45e1-4d21-8809-a40b5b8f7ef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:54.445158 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1419/2849] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:36:54 2022] DELETE /identity/v3/projects/3da2a386a837446dad4bf8b8506e418d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:54.450286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46f13-1b26-47e4-bea8-0c5e15ce5d73 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.450573 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46f13-1b26-47e4-bea8-0c5e15ce5d73 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.450906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46f13-1b26-47e4-bea8-0c5e15ce5d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.471365 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-24d46f13-1b26-47e4-bea8-0c5e15ce5d73 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:54.472563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-24d46f13-1b26-47e4-bea8-0c5e15ce5d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:54.530229 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1431/2850] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:36:54.538106 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410d61-a505-490a-b809-1b881545d7ba None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.538393 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410d61-a505-490a-b809-1b881545d7ba None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.538667 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410d61-a505-490a-b809-1b881545d7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.558034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-82410d61-a505-490a-b809-1b881545d7ba None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:54.559226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82410d61-a505-490a-b809-1b881545d7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:54.566893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1420/2851] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:54.571531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d46f13-1b26-47e4-bea8-0c5e15ce5d73 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.580933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d46f13-1b26-47e4-bea8-0c5e15ce5d73 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.586418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.586945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.587221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.587489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.588124 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:36:54.588429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.590021 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.601344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1432/2852] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 12:36:54.605529 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82410d61-a505-490a-b809-1b881545d7ba None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.614665 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82410d61-a505-490a-b809-1b881545d7ba None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.619304 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.619888 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.620164 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.620432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.621060 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.623056 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.650043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1421/2853] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:36:54 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:36:54.654230 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.664199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150bcb4-93be-4921-8c33-6c531b66863d admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.668621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:54.727275 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.727887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.728165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.728434 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.729114 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.731039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.752550 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1433/2854] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:36:54 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:36:54.757316 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.767252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78015cd9-5b00-46ff-b1ea-d90dd9d29c80 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.771659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.772243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.773157 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.773448 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.774096 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.775659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.785218 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1422/2855] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:36:54.788937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.798714 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4f47-477a-4c2f-b33b-e4b14189f559 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.803290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cca6b-48ff-442b-b815-bbbd6cfcb39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.803914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cca6b-48ff-442b-b815-bbbd6cfcb39c admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.804193 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cca6b-48ff-442b-b815-bbbd6cfcb39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.804469 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cca6b-48ff-442b-b815-bbbd6cfcb39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.805109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cca6b-48ff-442b-b815-bbbd6cfcb39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.806760 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cca6b-48ff-442b-b815-bbbd6cfcb39c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.818352 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1434/2856] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:36:54.823280 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.833358 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd6650-9230-41fd-8b65-629d51622853 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:54.839030 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:54.839606 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.839887 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.840160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] PATH_INFO: `/v3/projects/6bb6cdd23a9d4a35bd373b4a45990df8/users/ac7db45c44554c4a9b134da35c5b52d2/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.854347 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb6cdd23a9d4a35bd373b4a45990df8, user_id=ac7db45c44554c4a9b134da35c5b52d2, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:54.857047 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:54.870751 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1423/2857] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:36:54 2022] PUT /identity/v3/projects/6bb6cdd23a9d4a35bd373b4a45990df8/users/ac7db45c44554c4a9b134da35c5b52d2/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:36:54.880711 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eee63f-4964-4ca2-b121-2bdc2aca8bf6 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:54.880998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eee63f-4964-4ca2-b121-2bdc2aca8bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:54.881276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eee63f-4964-4ca2-b121-2bdc2aca8bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:54.909489 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-d9eee63f-4964-4ca2-b121-2bdc2aca8bf6 None None] MFA Rules not processed for user `ac7db45c44554c4a9b134da35c5b52d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:54.910750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-d9eee63f-4964-4ca2-b121-2bdc2aca8bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:54.975254 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1435/2858] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:54.979801 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:54.997784 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:55.048268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8ab3-1c20-4317-891f-b49c331f8822 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.052870 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.053471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.054282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.054554 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.055188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.057071 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.084546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1424/2859] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:36:54 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:36:55.088882 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eee63f-4964-4ca2-b121-2bdc2aca8bf6 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:55.099826 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eee63f-4964-4ca2-b121-2bdc2aca8bf6 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.155598 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.156207 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.156486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.156756 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.157376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.159328 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.180551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1436/2860] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:36:55 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:36:55.184532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:55.195231 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618f02b-4cac-48b8-b195-43b7c400a54d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.199759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.200354 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.200634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.200906 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.201546 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.203172 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.213927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1425/2861] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:36:55.217906 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:55.224008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:55.228996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462a7d9-2df0-43ba-9b8f-fea54adaf751 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.234732 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.235389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.235670 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.235943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] PATH_INFO: `/v3/projects/50b5273663ab48928d3e7dc784e4bd32/users/feb58de9b8d643b1a6c3651b07ca4116/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.252513 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b5273663ab48928d3e7dc784e4bd32, user_id=feb58de9b8d643b1a6c3651b07ca4116, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.255221 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.269903 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1437/2862] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:36:55 2022] PUT /identity/v3/projects/50b5273663ab48928d3e7dc784e4bd32/users/feb58de9b8d643b1a6c3651b07ca4116/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:36:55.273966 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:55.284450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b85754-b21a-4bf0-9758-b26e91b13061 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.340433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276acac2-f26d-4952-8121-a8d636de4918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.341028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276acac2-f26d-4952-8121-a8d636de4918 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.341305 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276acac2-f26d-4952-8121-a8d636de4918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.341573 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276acac2-f26d-4952-8121-a8d636de4918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.342243 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276acac2-f26d-4952-8121-a8d636de4918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.343813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276acac2-f26d-4952-8121-a8d636de4918 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.353995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1426/2863] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:36:55.358145 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:55.368715 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e58b8-f613-4aea-8791-e912c012bc3a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.373347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.374020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.374300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.374599 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] PATH_INFO: `/v3/projects/50b5273663ab48928d3e7dc784e4bd32/users/feb58de9b8d643b1a6c3651b07ca4116/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.376500 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b5273663ab48928d3e7dc784e4bd32, user_id=feb58de9b8d643b1a6c3651b07ca4116, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.379216 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.389174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1438/2864] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:36:55 2022] PUT /identity/v3/projects/50b5273663ab48928d3e7dc784e4bd32/users/feb58de9b8d643b1a6c3651b07ca4116/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:55.399989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01c1f-c69e-41fd-831a-81636843a9d3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.400269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01c1f-c69e-41fd-831a-81636843a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.400538 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01c1f-c69e-41fd-831a-81636843a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.427946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-84c01c1f-c69e-41fd-831a-81636843a9d3 None None] MFA Rules not processed for user `feb58de9b8d643b1a6c3651b07ca4116`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:55.429724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-84c01c1f-c69e-41fd-831a-81636843a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:55.495642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1427/2865] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:55.506999 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:55.508778 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:55.568763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fae488-554f-4920-aac5-fb2d8f2b663b admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.572436 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.573045 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.573326 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.573601 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.582934 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.584347 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.594610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1439/2866] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:55.672412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c01c1f-c69e-41fd-831a-81636843a9d3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:55.682512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c01c1f-c69e-41fd-831a-81636843a9d3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:55.686195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983db17-e3d3-42e1-869a-e0ef31c31694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:55.686789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983db17-e3d3-42e1-869a-e0ef31c31694 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:55.687070 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983db17-e3d3-42e1-869a-e0ef31c31694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:55.687344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983db17-e3d3-42e1-869a-e0ef31c31694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:55.692458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:55.745222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983db17-e3d3-42e1-869a-e0ef31c31694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:55.746672 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983db17-e3d3-42e1-869a-e0ef31c31694 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:55.755548 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1428/2867] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:58.464786 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:58.525757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a6c7-eb2b-40b7-90d3-a074607cf7fe service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:58.530263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:58.530880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:58.531156 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:58.531424 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] PATH_INFO: `/v3/users/ac7db45c44554c4a9b134da35c5b52d2` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:58.532526 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7db45c44554c4a9b134da35c5b52d2)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:58.534471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:58.578033 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7db45c44554c4a9b134da35c5b52d2'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:58.585953 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7db45c44554c4a9b134da35c5b52d2'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:58.586651 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7db45c44554c4a9b134da35c5b52d2'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:58.593456 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7db45c44554c4a9b134da35c5b52d2'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:58.600423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:58.606983 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7db45c44554c4a9b134da35c5b52d2'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:58.613238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1440/2868] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:36:58 2022] DELETE /identity/v3/users/ac7db45c44554c4a9b134da35c5b52d2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:58.618412 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:58.619176 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:58.620224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:58.639432 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:58.640704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:58.698502 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1429/2869] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:58.708032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:58.709423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:58.765820 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e603b0-e6cb-4958-af38-e751f8a77588 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:58.769466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:58.770088 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:58.770372 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:58.770645 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:58.779704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:58.781180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:58.790641 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1441/2870] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:36:59.065693 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:59.067191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:59.085485 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed6ace-8a2d-4bee-bf11-169fcd01cb27 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:59.090753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:59.091568 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:59.091893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:59.092249 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] PATH_INFO: `/v3/projects/6bb6cdd23a9d4a35bd373b4a45990df8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:59.093626 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb6cdd23a9d4a35bd373b4a45990df8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:59.095650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:59.146996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb6cdd23a9d4a35bd373b4a45990df8', 'request_id': 'req-98989188-7bf9-476d-9693-28a7d6b5e369'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.154169 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb6cdd23a9d4a35bd373b4a45990df8', 'request_id': 'req-98989188-7bf9-476d-9693-28a7d6b5e369'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.159709 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb6cdd23a9d4a35bd373b4a45990df8', 'request_id': 'req-98989188-7bf9-476d-9693-28a7d6b5e369'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.172366 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] The token cache is being invalidate because project 6bb6cdd23a9d4a35bd373b4a45990df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:36:59.172655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.173265 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.174809 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1430/2871] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:36:59 2022] DELETE /identity/v3/projects/6bb6cdd23a9d4a35bd373b4a45990df8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:59.179482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:59.189057 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e9c87-99a7-47a1-a696-f7fec4ad8b5f service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:59.244455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:59.245008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:59.245290 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:59.245563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] PATH_INFO: `/v3/users/feb58de9b8d643b1a6c3651b07ca4116` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:59.246712 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb58de9b8d643b1a6c3651b07ca4116)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:59.248652 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:59.292102 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb58de9b8d643b1a6c3651b07ca4116'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.298878 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb58de9b8d643b1a6c3651b07ca4116'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.299529 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb58de9b8d643b1a6c3651b07ca4116'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.306950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb58de9b8d643b1a6c3651b07ca4116'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.318656 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d29820dd-52a0-4d6d-8ab8-52b8a4fb57fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb58de9b8d643b1a6c3651b07ca4116'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.324828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1442/2872] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:36:59 2022] DELETE /identity/v3/users/feb58de9b8d643b1a6c3651b07ca4116 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:36:59.591541 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:59.592971 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:59.611014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98989188-7bf9-476d-9693-28a7d6b5e369 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:59.641385 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:59.669512 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:59.670171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:59.670450 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:59.670722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] PATH_INFO: `/v3/projects/50b5273663ab48928d3e7dc784e4bd32` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:59.671829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b5273663ab48928d3e7dc784e4bd32)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:59.673713 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:59.723043 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b5273663ab48928d3e7dc784e4bd32', 'request_id': 'req-5a183018-73f3-4b62-a945-44054d8b5c79'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.729642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b5273663ab48928d3e7dc784e4bd32', 'request_id': 'req-5a183018-73f3-4b62-a945-44054d8b5c79'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.735264 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b5273663ab48928d3e7dc784e4bd32', 'request_id': 'req-5a183018-73f3-4b62-a945-44054d8b5c79'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.747519 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] The token cache is being invalidate because project 50b5273663ab48928d3e7dc784e4bd32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:36:59.747802 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.748509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5a183018-73f3-4b62-a945-44054d8b5c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:36:59.750007 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1431/2873] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:36:59 2022] DELETE /identity/v3/projects/50b5273663ab48928d3e7dc784e4bd32 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:36:59.755561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873b20-c8b4-4a9b-8a86-e11e4a1e8296 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:59.756020 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873b20-c8b4-4a9b-8a86-e11e4a1e8296 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:59.756113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873b20-c8b4-4a9b-8a86-e11e4a1e8296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:59.776164 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-8f873b20-c8b4-4a9b-8a86-e11e4a1e8296 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:59.777266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-8f873b20-c8b4-4a9b-8a86-e11e4a1e8296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:59.782488 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:36:59.836791 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1443/2874] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:59.844348 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc8e6-c8e1-4d95-bd95-ee129d107408 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:59.844643 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc8e6-c8e1-4d95-bd95-ee129d107408 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:59.844907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc8e6-c8e1-4d95-bd95-ee129d107408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:59.864438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bcffc8e6-c8e1-4d95-bd95-ee129d107408 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:36:59.865627 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bcffc8e6-c8e1-4d95-bd95-ee129d107408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:36:59.873892 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1432/2875] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:36:59.877896 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f873b20-c8b4-4a9b-8a86-e11e4a1e8296 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:59.887572 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f873b20-c8b4-4a9b-8a86-e11e4a1e8296 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:59.892104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:59.892692 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:59.892972 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:59.893246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:59.893943 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:36:59.894245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:59.895816 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:59.907130 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1444/2876] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:36:59.911252 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffc8e6-c8e1-4d95-bd95-ee129d107408 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:59.920785 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffc8e6-c8e1-4d95-bd95-ee129d107408 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:36:59.925904 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:36:59.926741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:36:59.927046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:36:59.927318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:36:59.927954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:36:59.930480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:36:59.956736 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1433/2877] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:36:59 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:36:59.961343 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:36:59.972829 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb8642-ffc8-4264-b64f-03db005591c2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.028439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.028970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.029249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.029519 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.030176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.032067 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.052590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1445/2878] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:36:59 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:00.057324 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.067458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861161c-461e-4a30-9244-63272ea7e0c0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.072015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.072608 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.072889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.073160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.073834 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.075415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.086075 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1434/2879] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:37:00.089839 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.099474 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c560d6-396a-4076-908f-615c5a8cfbeb admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.104866 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9612ac8-7d13-4720-bf6f-8cf2d433d757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.105453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612ac8-7d13-4720-bf6f-8cf2d433d757 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.105729 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612ac8-7d13-4720-bf6f-8cf2d433d757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.106028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612ac8-7d13-4720-bf6f-8cf2d433d757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.106668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9612ac8-7d13-4720-bf6f-8cf2d433d757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.108266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9612ac8-7d13-4720-bf6f-8cf2d433d757 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.118554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1446/2880] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:00.122618 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.133142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9c440-9440-4a36-ad0d-ad034f85d512 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.137726 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.138404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.138683 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.138958 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] PATH_INFO: `/v3/projects/b36d474208364fd1827b02d86398f380/users/29fe0ae5ec0544948c50ea8fc75f18cf/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.153954 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36d474208364fd1827b02d86398f380, user_id=29fe0ae5ec0544948c50ea8fc75f18cf, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.156761 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.170367 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1435/2881] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:00 2022] PUT /identity/v3/projects/b36d474208364fd1827b02d86398f380/users/29fe0ae5ec0544948c50ea8fc75f18cf/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:00.181848 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eb457-a3f6-4512-a460-46b9c36eb1ea None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.182197 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eb457-a3f6-4512-a460-46b9c36eb1ea None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.182408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eb457-a3f6-4512-a460-46b9c36eb1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.211222 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-183eb457-a3f6-4512-a460-46b9c36eb1ea None None] MFA Rules not processed for user `29fe0ae5ec0544948c50ea8fc75f18cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:00.212440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-183eb457-a3f6-4512-a460-46b9c36eb1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:00.278054 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1447/2882] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:00.282717 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.342829 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:00.346956 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82809eea-3899-47fa-abf7-0ca9912c59f3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.354018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.354646 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.355079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.355356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.355984 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.357907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.383413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1436/2883] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:37:00 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:00.387374 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183eb457-a3f6-4512-a460-46b9c36eb1ea None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.398498 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183eb457-a3f6-4512-a460-46b9c36eb1ea None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.402042 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:00.456681 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.457274 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.457549 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.457843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.458472 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.461201 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.482539 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1448/2884] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:37:00 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:00.486537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.496698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a112a-faee-4448-91ce-21bd2bf45cd1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.501363 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.501986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.502263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.502530 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.503160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.504722 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.515658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1437/2885] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:00.519561 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.530166 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c170cde-e9cb-4b36-a357-561ec359e5d7 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.534796 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.535389 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.535668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.535944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] PATH_INFO: `/v3/projects/2bef2ef0f9674cbf847897905a60978f/users/4419279f2354447fb946b6d91373cfd8/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.550335 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bef2ef0f9674cbf847897905a60978f, user_id=4419279f2354447fb946b6d91373cfd8, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.552955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.565492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1449/2886] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:00 2022] PUT /identity/v3/projects/2bef2ef0f9674cbf847897905a60978f/users/4419279f2354447fb946b6d91373cfd8/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:37:00.569941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.579766 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db33ff6-ced6-4d70-876d-942471db6c2c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.641986 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbf7d6-9fed-41b3-9816-8ef37dde5822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.642612 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbf7d6-9fed-41b3-9816-8ef37dde5822 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.642902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbf7d6-9fed-41b3-9816-8ef37dde5822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.643133 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbf7d6-9fed-41b3-9816-8ef37dde5822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.643781 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbf7d6-9fed-41b3-9816-8ef37dde5822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.645401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbf7d6-9fed-41b3-9816-8ef37dde5822 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.656237 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1438/2887] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:00.660369 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.670510 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ab2a3-8a68-4042-aeb3-4aaf574f38bb admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.675051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.675706 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.675982 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.676253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] PATH_INFO: `/v3/projects/2bef2ef0f9674cbf847897905a60978f/users/4419279f2354447fb946b6d91373cfd8/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.678176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bef2ef0f9674cbf847897905a60978f, user_id=4419279f2354447fb946b6d91373cfd8, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.680888 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.691090 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1450/2888] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:00 2022] PUT /identity/v3/projects/2bef2ef0f9674cbf847897905a60978f/users/4419279f2354447fb946b6d91373cfd8/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:00.701650 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2ba4c-c148-4b71-a11f-b3538d3b0458 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.702055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2ba4c-c148-4b71-a11f-b3538d3b0458 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.702320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2ba4c-c148-4b71-a11f-b3538d3b0458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.730411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-61d2ba4c-c148-4b71-a11f-b3538d3b0458 None None] MFA Rules not processed for user `4419279f2354447fb946b6d91373cfd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:00.731630 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-61d2ba4c-c148-4b71-a11f-b3538d3b0458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:00.801581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1439/2889] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:00.813168 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:00.815011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:00.872273 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3331-f036-4f81-92b4-dc9039b72688 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:00.876103 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:00.876699 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:00.876976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:00.877246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:00.886194 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:00.887618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:00.896744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1451/2890] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:01.003955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2ba4c-c148-4b71-a11f-b3538d3b0458 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:01.013843 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2ba4c-c148-4b71-a11f-b3538d3b0458 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:01.017909 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563908ab-bd4f-477b-b597-46e4a2e5d163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:01.018523 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563908ab-bd4f-477b-b597-46e4a2e5d163 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:01.018804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563908ab-bd4f-477b-b597-46e4a2e5d163 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:01.019080 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563908ab-bd4f-477b-b597-46e4a2e5d163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:01.072741 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:01.083930 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563908ab-bd4f-477b-b597-46e4a2e5d163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:01.086199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563908ab-bd4f-477b-b597-46e4a2e5d163 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:01.096028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1440/2891] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:03.701482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:03.772154 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb797bb-6ce7-4155-bb60-8d3764f5a138 service cinder] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:03.776815 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:03.777446 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:03.777750 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:03.778165 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] PATH_INFO: `/v3/users/29fe0ae5ec0544948c50ea8fc75f18cf` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:03.779551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29fe0ae5ec0544948c50ea8fc75f18cf)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:03.781444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:03.783998 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:03.841607 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29fe0ae5ec0544948c50ea8fc75f18cf'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:03.849575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29fe0ae5ec0544948c50ea8fc75f18cf'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:03.850466 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29fe0ae5ec0544948c50ea8fc75f18cf'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:03.858317 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29fe0ae5ec0544948c50ea8fc75f18cf'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:03.870508 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29fe0ae5ec0544948c50ea8fc75f18cf'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:03.876420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1452/2892] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:37:03 2022] DELETE /identity/v3/users/29fe0ae5ec0544948c50ea8fc75f18cf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:03.880929 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:03.881435 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:03.881552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:03.902357 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:03.903550 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:03.961333 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1441/2893] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:37:03.971107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:03.972486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:04.039938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f063-d151-4fb8-80c8-8c0491982ef7 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:04.043703 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:04.044311 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:04.044592 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:04.044863 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:04.054223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:04.055640 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:04.065176 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1453/2894] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:04.329557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:04.331220 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:04.344120 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d036708-e57a-4b1f-b043-c33dc3c54149 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:04.348697 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:04.349417 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:04.349778 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:04.350151 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] PATH_INFO: `/v3/projects/b36d474208364fd1827b02d86398f380` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:04.351480 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36d474208364fd1827b02d86398f380)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:04.353738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:04.400750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36d474208364fd1827b02d86398f380', 'request_id': 'req-a5968208-d2aa-41d2-a497-08640b524c5e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.406753 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36d474208364fd1827b02d86398f380', 'request_id': 'req-a5968208-d2aa-41d2-a497-08640b524c5e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.412222 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36d474208364fd1827b02d86398f380', 'request_id': 'req-a5968208-d2aa-41d2-a497-08640b524c5e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.424113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] The token cache is being invalidate because project b36d474208364fd1827b02d86398f380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:37:04.424386 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.424999 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.426494 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1442/2895] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:37:04 2022] DELETE /identity/v3/projects/b36d474208364fd1827b02d86398f380 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:04.430610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:04.440074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259e936-51d8-4e6c-b8a4-9a34ee406e89 service neutron] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:04.495944 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:04.496476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:04.496749 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:04.497016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] PATH_INFO: `/v3/users/4419279f2354447fb946b6d91373cfd8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:04.498133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4419279f2354447fb946b6d91373cfd8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:04.500011 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:04.527107 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:04.545874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4419279f2354447fb946b6d91373cfd8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.552946 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4419279f2354447fb946b6d91373cfd8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.553627 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4419279f2354447fb946b6d91373cfd8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.560939 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4419279f2354447fb946b6d91373cfd8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.573038 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-9d2c1267-c9c8-45ac-9241-36504b103eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4419279f2354447fb946b6d91373cfd8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.579476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1454/2896] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:37:04 2022] DELETE /identity/v3/users/4419279f2354447fb946b6d91373cfd8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:04.845320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:04.846771 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:04.860123 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5968208-d2aa-41d2-a497-08640b524c5e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:04.915830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:04.916471 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:04.916723 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:04.916995 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] PATH_INFO: `/v3/projects/2bef2ef0f9674cbf847897905a60978f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:04.918235 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bef2ef0f9674cbf847897905a60978f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:04.920168 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:04.939193 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:04.975334 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bef2ef0f9674cbf847897905a60978f', 'request_id': 'req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.982442 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bef2ef0f9674cbf847897905a60978f', 'request_id': 'req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:04.988015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bef2ef0f9674cbf847897905a60978f', 'request_id': 'req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:05.002522 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] The token cache is being invalidate because project 2bef2ef0f9674cbf847897905a60978f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:37:05.002813 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:05.004091 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cbe44d64-8ffc-48e6-98b9-f12fdf30a177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:05.006345 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1443/2897] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:37:04 2022] DELETE /identity/v3/projects/2bef2ef0f9674cbf847897905a60978f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:05.010950 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccfeb9-a477-4673-86ef-e7e6d815c3f9 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.011449 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccfeb9-a477-4673-86ef-e7e6d815c3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.011553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccfeb9-a477-4673-86ef-e7e6d815c3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.033127 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-37ccfeb9-a477-4673-86ef-e7e6d815c3f9 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:05.034269 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-37ccfeb9-a477-4673-86ef-e7e6d815c3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:05.100830 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1455/2898] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:05.110079 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032fcfe-6c3e-42a5-b4a4-8d1931425252 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.110217 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032fcfe-6c3e-42a5-b4a4-8d1931425252 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.110563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032fcfe-6c3e-42a5-b4a4-8d1931425252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.138230 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4032fcfe-6c3e-42a5-b4a4-8d1931425252 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:05.139458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4032fcfe-6c3e-42a5-b4a4-8d1931425252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:05.150048 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1444/2899] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:05.154332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ccfeb9-a477-4673-86ef-e7e6d815c3f9 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.167031 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ccfeb9-a477-4673-86ef-e7e6d815c3f9 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.171751 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.172364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.172648 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.172924 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.173575 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:37:05.173915 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.175541 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.191267 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1456/2900] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:37:05.192381 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032fcfe-6c3e-42a5-b4a4-8d1931425252 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.201334 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032fcfe-6c3e-42a5-b4a4-8d1931425252 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.205927 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.206516 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.206787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.207055 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.207682 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.209601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.240680 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1445/2901] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:37:05 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:37:05.242632 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.254441 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09034208-e672-4952-b85c-209cf490f2ee admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.309822 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.311526 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:05.313975 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.314586 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.314868 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.315144 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] PATH_INFO: `/v3/users` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.315769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.317678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.342286 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1457/2902] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:37:05 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:05.343322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.353476 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c56be2-eb55-4486-89b6-9d7dee6e9e5a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.359117 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.359719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.360009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.360278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.360914 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.362524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.371806 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3670-551d-4a81-9bb6-93ed65eb53f3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.373180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1458/2903] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:37:05.375676 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ac9a1-ba38-4ab4-901d-91c42fcb4248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.376279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ac9a1-ba38-4ab4-901d-91c42fcb4248 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.376563 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ac9a1-ba38-4ab4-901d-91c42fcb4248 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.376839 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ac9a1-ba38-4ab4-901d-91c42fcb4248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.377266 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.378658 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f40ac9a1-ba38-4ab4-901d-91c42fcb4248 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:05.386892 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d82fa-36aa-45ef-9a89-d0061e2ee9d0 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.391636 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.392276 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.392555 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.392824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.393453 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.395043 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.397085 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:05.416585 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1459/2904] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:37:05.418527 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.430444 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfccad-a2b6-4024-8b56-06bf7ebb67a9 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.435272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.435925 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.436204 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.436479 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] PATH_INFO: `/v3/projects/3c6c148d47fb47a8abcfa7ec41a628d8/users/00c3819af005409cb29ad04a8bb15e20/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.445171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ac9a1-ba38-4ab4-901d-91c42fcb4248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.446629 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ac9a1-ba38-4ab4-901d-91c42fcb4248 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.450859 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6c148d47fb47a8abcfa7ec41a628d8, user_id=00c3819af005409cb29ad04a8bb15e20, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.453492 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.456171 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1446/2905] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:37:05.467440 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1460/2906] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:05 2022] PUT /identity/v3/projects/3c6c148d47fb47a8abcfa7ec41a628d8/users/00c3819af005409cb29ad04a8bb15e20/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:05.479364 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72c0fb-315b-4f73-acb6-2e27245b159f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.479649 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72c0fb-315b-4f73-acb6-2e27245b159f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.479919 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72c0fb-315b-4f73-acb6-2e27245b159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.507749 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ab72c0fb-315b-4f73-acb6-2e27245b159f None None] MFA Rules not processed for user `00c3819af005409cb29ad04a8bb15e20`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:05.508941 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ab72c0fb-315b-4f73-acb6-2e27245b159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:05.522020 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:05.577183 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1447/2907] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:05.581593 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.645613 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b24cad-c3b7-40b8-8848-b2f49ec14a0c admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.650735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.651345 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.651630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.651910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.652540 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.654516 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.685191 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1461/2908] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:37:05 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:05.686402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72c0fb-315b-4f73-acb6-2e27245b159f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.697928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72c0fb-315b-4f73-acb6-2e27245b159f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.759415 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.760029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.760389 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.760581 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.761234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.763173 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.786768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1448/2909] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:37:05 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:37:05.790742 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.801327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346a08-1a89-4590-8a89-f99ce51cb985 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.806113 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.806725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.807006 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.807278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.807929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.809549 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.821880 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1462/2910] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:37:05.825542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.836344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2f419-758e-41e1-94a8-344a71b58696 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.841791 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.842461 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.842748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.843015 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] PATH_INFO: `/v3/projects/40d79bf8be0c4861afee439c6e2e0f1c/users/bcb0dda000764c46bf4ea05fe58fb768/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.859050 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d79bf8be0c4861afee439c6e2e0f1c, user_id=bcb0dda000764c46bf4ea05fe58fb768, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.861692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.875310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1449/2911] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:05 2022] PUT /identity/v3/projects/40d79bf8be0c4861afee439c6e2e0f1c/users/bcb0dda000764c46bf4ea05fe58fb768/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:05.879595 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.890415 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cb77-c8f8-4177-8c7f-9388e3fb08cc admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.951604 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6edd57-340c-4f78-bde1-9aaf381b0856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.952255 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6edd57-340c-4f78-bde1-9aaf381b0856 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.952535 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6edd57-340c-4f78-bde1-9aaf381b0856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.952811 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6edd57-340c-4f78-bde1-9aaf381b0856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.953465 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6edd57-340c-4f78-bde1-9aaf381b0856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.955089 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6edd57-340c-4f78-bde1-9aaf381b0856 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:05.966379 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1463/2912] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:37:05.970609 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:05.980855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564635ad-3ea1-4a20-9ffa-4f62df539148 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:05.986196 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:05.986851 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:05.987131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:05.987404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] PATH_INFO: `/v3/projects/40d79bf8be0c4861afee439c6e2e0f1c/users/bcb0dda000764c46bf4ea05fe58fb768/roles/0b2bdba44e3544f18a9d494b1acc8711` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:05.989799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d79bf8be0c4861afee439c6e2e0f1c, user_id=bcb0dda000764c46bf4ea05fe58fb768, role_id=0b2bdba44e3544f18a9d494b1acc8711)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:05.991967 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:06.002497 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1450/2913] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:05 2022] PUT /identity/v3/projects/40d79bf8be0c4861afee439c6e2e0f1c/users/bcb0dda000764c46bf4ea05fe58fb768/roles/0b2bdba44e3544f18a9d494b1acc8711 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:06.014077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c4100-7d5c-4c69-8441-d841c22999cc None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:06.014615 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c4100-7d5c-4c69-8441-d841c22999cc None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:06.014911 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c4100-7d5c-4c69-8441-d841c22999cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:06.044976 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-b93c4100-7d5c-4c69-8441-d841c22999cc None None] MFA Rules not processed for user `bcb0dda000764c46bf4ea05fe58fb768`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:06.046249 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-b93c4100-7d5c-4c69-8441-d841c22999cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:06.082008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:06.119668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1464/2914] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:06.134505 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:06.136339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:06.195321 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6e3a6-e8ce-4658-a041-2a98d956e101 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:06.198804 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:06.199401 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:06.199675 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:06.199940 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:06.209269 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:06.211307 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:06.221014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1451/2915] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:06.310237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c4100-7d5c-4c69-8441-d841c22999cc None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:06.320736 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c4100-7d5c-4c69-8441-d841c22999cc None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:06.324579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497a2b9-6534-4b9f-b2ac-9c410ed08e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:06.325251 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497a2b9-6534-4b9f-b2ac-9c410ed08e00 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:06.325653 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497a2b9-6534-4b9f-b2ac-9c410ed08e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:06.325949 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497a2b9-6534-4b9f-b2ac-9c410ed08e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:06.385671 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497a2b9-6534-4b9f-b2ac-9c410ed08e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:06.387093 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497a2b9-6534-4b9f-b2ac-9c410ed08e00 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:06.396909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1465/2916] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:11.122459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:11.135935 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:11.186006 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04943101-f839-4c52-a73e-ea4c39353399 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:11.190509 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:11.191131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:11.191413 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:11.191687 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] PATH_INFO: `/v3/users/00c3819af005409cb29ad04a8bb15e20` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:11.192786 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c3819af005409cb29ad04a8bb15e20)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:11.194799 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:11.240594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c3819af005409cb29ad04a8bb15e20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.241154 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c3819af005409cb29ad04a8bb15e20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.248446 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c3819af005409cb29ad04a8bb15e20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.255495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c3819af005409cb29ad04a8bb15e20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.268114 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c3819af005409cb29ad04a8bb15e20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.274152 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1452/2917] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:37:11 2022] DELETE /identity/v3/users/00c3819af005409cb29ad04a8bb15e20 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:11.278725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:11.279178 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:11.279277 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:11.298300 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:11.299486 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:11.357189 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1466/2918] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:37:11.366853 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:11.368896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:11.425724 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20842463-413d-41bd-b3c1-6bd7aeafb29d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:11.429356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:11.429989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:11.430273 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:11.430549 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:11.439989 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:11.441405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:11.450896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1453/2919] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:11.725481 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:11.727081 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:11.742576 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774ce3e-60c3-4bff-94af-e357406d89fc None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:11.747419 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:11.748053 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:11.748324 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:11.748621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] PATH_INFO: `/v3/projects/3c6c148d47fb47a8abcfa7ec41a628d8` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:11.749970 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6c148d47fb47a8abcfa7ec41a628d8)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:11.751803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:11.797824 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6c148d47fb47a8abcfa7ec41a628d8', 'request_id': 'req-5964817b-953c-435b-988d-9f956cc3e6ef'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.805491 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6c148d47fb47a8abcfa7ec41a628d8', 'request_id': 'req-5964817b-953c-435b-988d-9f956cc3e6ef'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.812186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6c148d47fb47a8abcfa7ec41a628d8', 'request_id': 'req-5964817b-953c-435b-988d-9f956cc3e6ef'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.814226 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:11.828675 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] The token cache is being invalidate because project 3c6c148d47fb47a8abcfa7ec41a628d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:37:11.828989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.829910 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.832051 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1467/2920] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:37:11 2022] DELETE /identity/v3/projects/3c6c148d47fb47a8abcfa7ec41a628d8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:11.836601 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:11.847513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a865d-33ee-4f5c-aeb8-4ee9fee57f62 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:11.876731 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:11.905156 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:11.905821 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:11.906233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:11.906527 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] PATH_INFO: `/v3/users/bcb0dda000764c46bf4ea05fe58fb768` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:11.907661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb0dda000764c46bf4ea05fe58fb768)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:11.909558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:11.954453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb0dda000764c46bf4ea05fe58fb768'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.955100 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb0dda000764c46bf4ea05fe58fb768'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.962759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb0dda000764c46bf4ea05fe58fb768'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.970013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb0dda000764c46bf4ea05fe58fb768'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.982533 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f1c440a-6cf5-49af-89d7-39e38e6bbb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb0dda000764c46bf4ea05fe58fb768'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:11.988188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1454/2921] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:37:11 2022] DELETE /identity/v3/users/bcb0dda000764c46bf4ea05fe58fb768 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:12.264027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.265471 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:12.279278 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964817b-953c-435b-988d-9f956cc3e6ef admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:12.335522 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:12.336333 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.336608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.336874 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] PATH_INFO: `/v3/projects/40d79bf8be0c4861afee439c6e2e0f1c` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.338662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d79bf8be0c4861afee439c6e2e0f1c)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:12.340531 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:12.384602 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d79bf8be0c4861afee439c6e2e0f1c', 'request_id': 'req-ce661b1f-2046-4c1b-bf90-f8786cc147bc'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:12.391199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d79bf8be0c4861afee439c6e2e0f1c', 'request_id': 'req-ce661b1f-2046-4c1b-bf90-f8786cc147bc'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:12.397344 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d79bf8be0c4861afee439c6e2e0f1c', 'request_id': 'req-ce661b1f-2046-4c1b-bf90-f8786cc147bc'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:12.408044 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] The token cache is being invalidate because project 40d79bf8be0c4861afee439c6e2e0f1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:37:12.408334 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:12.408961 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-ce661b1f-2046-4c1b-bf90-f8786cc147bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:12.410450 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1468/2922] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:37:12 2022] DELETE /identity/v3/projects/40d79bf8be0c4861afee439c6e2e0f1c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:12.415397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec20a-7726-4ce0-9930-d5b79ac2dda8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.415876 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec20a-7726-4ce0-9930-d5b79ac2dda8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.415973 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec20a-7726-4ce0-9930-d5b79ac2dda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.436132 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f39ec20a-7726-4ce0-9930-d5b79ac2dda8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:12.437318 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f39ec20a-7726-4ce0-9930-d5b79ac2dda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:12.494279 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1455/2923] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:12.501821 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5529da6-caac-4e00-8c4b-0fe3f16e4e91 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.502109 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5529da6-caac-4e00-8c4b-0fe3f16e4e91 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.502384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5529da6-caac-4e00-8c4b-0fe3f16e4e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.522543 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-e5529da6-caac-4e00-8c4b-0fe3f16e4e91 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:12.523773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-e5529da6-caac-4e00-8c4b-0fe3f16e4e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:12.531886 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1469/2924] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:37:12.536000 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec20a-7726-4ce0-9930-d5b79ac2dda8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.545411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec20a-7726-4ce0-9930-d5b79ac2dda8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:12.549922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:12.550507 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.550783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.551052 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.551688 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:37:12.551992 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:12.553557 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:12.565880 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1456/2925] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:37:12.570055 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5529da6-caac-4e00-8c4b-0fe3f16e4e91 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.579758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5529da6-caac-4e00-8c4b-0fe3f16e4e91 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:12.584388 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:12.585005 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.585283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.585870 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.586463 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:12.588322 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:12.614390 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1470/2926] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:37:12 2022] POST /identity/v3/projects => generated 361 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:37:12.618923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.628755 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2be072-bc15-4dab-9188-9fbb0cf27014 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:12.666188 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:12.686263 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:12.686859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.687133 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.687399 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.688022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:12.689925 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:12.711104 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1457/2927] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:37:12 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:12.715272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.725323 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcbe84-1041-437c-a5ba-4a5bcd05494a admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:12.730172 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:12.730769 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.731046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.731316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.731951 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:12.733579 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:12.743923 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1471/2928] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:12.747812 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.757339 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c087fab-63e6-4c04-8278-ce139034ad7f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:12.762438 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235755a1-d558-4ae3-b9bc-de4c92c0f9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:12.763013 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235755a1-d558-4ae3-b9bc-de4c92c0f9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.763289 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235755a1-d558-4ae3-b9bc-de4c92c0f9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.763556 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235755a1-d558-4ae3-b9bc-de4c92c0f9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.764184 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235755a1-d558-4ae3-b9bc-de4c92c0f9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:12.765833 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235755a1-d558-4ae3-b9bc-de4c92c0f9d2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:12.775402 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1458/2929] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:37:12.779246 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.788753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eac723-7dea-4322-91cc-9826f4cd4316 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:12.793853 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:12.794482 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.794757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.795027 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] PATH_INFO: `/v3/projects/7858ff61097141958a1382f62844d802/users/1812031b4d074dc8a031add0a81edd25/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.809929 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7858ff61097141958a1382f62844d802, user_id=1812031b4d074dc8a031add0a81edd25, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:12.812538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:12.825622 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1472/2930] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:12 2022] PUT /identity/v3/projects/7858ff61097141958a1382f62844d802/users/1812031b4d074dc8a031add0a81edd25/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:37:12.837233 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d0ea2-9383-4567-a5a8-9edfdc6eea9e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:12.837486 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d0ea2-9383-4567-a5a8-9edfdc6eea9e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:12.837797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d0ea2-9383-4567-a5a8-9edfdc6eea9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:12.866394 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e71d0ea2-9383-4567-a5a8-9edfdc6eea9e None None] MFA Rules not processed for user `1812031b4d074dc8a031add0a81edd25`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:12.867586 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e71d0ea2-9383-4567-a5a8-9edfdc6eea9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:12.928286 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1459/2931] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:12.961958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:12.963678 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:12.984209 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:13.025360 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352ccbf-2c8c-4ff2-b078-fed3a6f071a0 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:13.028985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5f8c1-9942-43ea-b034-7a6dc2ae16ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:13.029588 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f8c1-9942-43ea-b034-7a6dc2ae16ac service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.029898 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f8c1-9942-43ea-b034-7a6dc2ae16ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.030174 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f8c1-9942-43ea-b034-7a6dc2ae16ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.038432 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5f8c1-9942-43ea-b034-7a6dc2ae16ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:13.039832 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5f8c1-9942-43ea-b034-7a6dc2ae16ac service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:13.049074 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1473/2932] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:13.065180 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d0ea2-9383-4567-a5a8-9edfdc6eea9e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:13.131765 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d0ea2-9383-4567-a5a8-9edfdc6eea9e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:13.136862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:13.137501 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.137803 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.138085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] PATH_INFO: `/v3/users/1812031b4d074dc8a031add0a81edd25` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.139228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1812031b4d074dc8a031add0a81edd25)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:13.141145 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:13.195294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1812031b4d074dc8a031add0a81edd25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.195937 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1812031b4d074dc8a031add0a81edd25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.198532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:13.204397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1812031b4d074dc8a031add0a81edd25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.212105 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1812031b4d074dc8a031add0a81edd25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.223850 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1812031b4d074dc8a031add0a81edd25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.229657 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1460/2933] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:37:13 2022] DELETE /identity/v3/users/1812031b4d074dc8a031add0a81edd25 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:13.234210 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.234630 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.234945 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.254167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:13.255412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:13.315635 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1474/2934] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:13.326293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:13.327772 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:13.383661 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c06aab-ca61-49da-b2d4-f502beaa5763 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:13.387750 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c52756-af80-4a1b-86ac-6cb1d911e120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:13.388353 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c52756-af80-4a1b-86ac-6cb1d911e120 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.388633 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c52756-af80-4a1b-86ac-6cb1d911e120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.388902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c52756-af80-4a1b-86ac-6cb1d911e120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.397953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c52756-af80-4a1b-86ac-6cb1d911e120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:13.399359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c52756-af80-4a1b-86ac-6cb1d911e120 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:13.408800 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1461/2935] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:13.678693 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:13.680237 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:13.695867 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2f5cb-20e0-465b-958f-5e8187ca60b2 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:13.700552 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:13.701182 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.701459 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.701730 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] PATH_INFO: `/v3/projects/7858ff61097141958a1382f62844d802` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.702875 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7858ff61097141958a1382f62844d802)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:13.704812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:13.753218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7858ff61097141958a1382f62844d802', 'request_id': 'req-3d01f035-edc5-47f0-8cbc-7c21f5667e58'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.760420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7858ff61097141958a1382f62844d802', 'request_id': 'req-3d01f035-edc5-47f0-8cbc-7c21f5667e58'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.766726 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7858ff61097141958a1382f62844d802', 'request_id': 'req-3d01f035-edc5-47f0-8cbc-7c21f5667e58'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.776782 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] The token cache is being invalidate because project 7858ff61097141958a1382f62844d802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:37:13.777071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.777704 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-3d01f035-edc5-47f0-8cbc-7c21f5667e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:13.779228 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1475/2936] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:37:13 2022] DELETE /identity/v3/projects/7858ff61097141958a1382f62844d802 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:13.784034 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f6e7b-a573-4248-b6d8-bc21c5abdc99 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.784496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f6e7b-a573-4248-b6d8-bc21c5abdc99 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.784591 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f6e7b-a573-4248-b6d8-bc21c5abdc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.803899 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ed7f6e7b-a573-4248-b6d8-bc21c5abdc99 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:13.805085 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ed7f6e7b-a573-4248-b6d8-bc21c5abdc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:13.864542 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1462/2937] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:37:13.872979 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627913-8aee-41e8-b233-60554e751dcc None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.873253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627913-8aee-41e8-b233-60554e751dcc None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.873528 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627913-8aee-41e8-b233-60554e751dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.892735 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-7b627913-8aee-41e8-b233-60554e751dcc None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:13.893957 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-7b627913-8aee-41e8-b233-60554e751dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:13.901663 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1476/2938] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:13.905754 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f6e7b-a573-4248-b6d8-bc21c5abdc99 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:13.915748 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f6e7b-a573-4248-b6d8-bc21c5abdc99 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:13.920241 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:13.920889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.921112 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.921411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.922083 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:37:13.922388 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:13.923991 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:13.935728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1463/2939] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:37:13.939808 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627913-8aee-41e8-b233-60554e751dcc None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:13.948807 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627913-8aee-41e8-b233-60554e751dcc None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:13.953366 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:13.953977 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:13.954253 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:13.954524 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:13.955153 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:13.957022 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:13.985028 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1477/2940] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:37:13 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:37:13.989144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:13.999018 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540444b-6ebb-4d5d-b938-72a87346e5f3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:14.050499 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:14.057202 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:14.058068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:14.058433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:14.058881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:14.059734 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:14.061859 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:14.082223 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1464/2941] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:37:13 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:14.086553 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:14.097104 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d28882-f914-40f7-91ff-2903d28ead4f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:14.101638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:14.102272 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:14.103152 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:14.103439 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:14.104078 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:14.105642 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:14.115353 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1478/2942] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:14.119199 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:14.131008 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449845bc-bc60-43d8-9f26-be5282041661 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:14.135659 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1c60d-b4f2-4309-8671-1db5ee670146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:14.136283 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1c60d-b4f2-4309-8671-1db5ee670146 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:14.136578 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1c60d-b4f2-4309-8671-1db5ee670146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:14.136862 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1c60d-b4f2-4309-8671-1db5ee670146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:14.137562 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1c60d-b4f2-4309-8671-1db5ee670146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:14.139182 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1c60d-b4f2-4309-8671-1db5ee670146 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:14.150423 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1465/2943] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:14.154505 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:14.160855 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:14.164594 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebb873-bc02-4731-96c4-483bbf682685 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:14.171560 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:14.172452 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:14.172907 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:14.173350 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] PATH_INFO: `/v3/projects/cc3c5da916694a38a26b1ac168048634/users/1f913b1519984563b4c7e45d2c9e98fc/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:14.188268 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc3c5da916694a38a26b1ac168048634, user_id=1f913b1519984563b4c7e45d2c9e98fc, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:14.190927 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:14.203218 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1479/2944] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:14 2022] PUT /identity/v3/projects/cc3c5da916694a38a26b1ac168048634/users/1f913b1519984563b4c7e45d2c9e98fc/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:37:14.215421 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddfdbb-d647-4c22-a6ca-9706e6fff5dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:14.215708 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddfdbb-d647-4c22-a6ca-9706e6fff5dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:14.216014 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddfdbb-d647-4c22-a6ca-9706e6fff5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:14.244431 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eaddfdbb-d647-4c22-a6ca-9706e6fff5dc None None] MFA Rules not processed for user `1f913b1519984563b4c7e45d2c9e98fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:14.245642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eaddfdbb-d647-4c22-a6ca-9706e6fff5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:14.307405 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1466/2945] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:14.318986 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:14.320668 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:14.376763 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcca32-ee84-4937-a388-c6125ee7a3b8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:14.380361 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24a097-1546-4510-a7c2-60d2bc64bfff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a24bc4a0c4f480192d87a27a243acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:14.380958 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24a097-1546-4510-a7c2-60d2bc64bfff service glance] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:14.381238 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24a097-1546-4510-a7c2-60d2bc64bfff service glance] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:14.381511 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24a097-1546-4510-a7c2-60d2bc64bfff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:14.390213 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24a097-1546-4510-a7c2-60d2bc64bfff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:14.391618 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24a097-1546-4510-a7c2-60d2bc64bfff service glance] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:14.400884 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1480/2946] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:24.304498 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:24.396356 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:46.713579 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddfdbb-d647-4c22-a6ca-9706e6fff5dc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:46.773881 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddfdbb-d647-4c22-a6ca-9706e6fff5dc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:46.778261 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:46.778902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:46.779185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:46.779459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] PATH_INFO: `/v3/users/1f913b1519984563b4c7e45d2c9e98fc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:46.780561 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f913b1519984563b4c7e45d2c9e98fc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:46.782537 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:46.828185 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f913b1519984563b4c7e45d2c9e98fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:46.828842 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f913b1519984563b4c7e45d2c9e98fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:46.837144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f913b1519984563b4c7e45d2c9e98fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:46.844513 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f913b1519984563b4c7e45d2c9e98fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:46.857189 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f913b1519984563b4c7e45d2c9e98fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:46.863453 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1467/2947] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:37:46 2022] DELETE /identity/v3/users/1f913b1519984563b4c7e45d2c9e98fc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:46.868037 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:46.868547 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:46.868664 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:46.887812 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:46.889004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:46.948128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1481/2948] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:46.958495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:46.959855 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:47.014953 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009458da-e1fc-4b7c-a051-935694d36511 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.019898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b85de7-27b3-4845-8572-c73e35fb5e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.020589 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85de7-27b3-4845-8572-c73e35fb5e45 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.020874 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85de7-27b3-4845-8572-c73e35fb5e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.021144 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85de7-27b3-4845-8572-c73e35fb5e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.030298 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b85de7-27b3-4845-8572-c73e35fb5e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.031694 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b85de7-27b3-4845-8572-c73e35fb5e45 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.040893 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1468/2949] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:47.305206 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.306657 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:47.319318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddd156-2843-44ac-9745-27ae2254a0c8 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.324996 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.325646 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.325967 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.326248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] PATH_INFO: `/v3/projects/cc3c5da916694a38a26b1ac168048634` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.327838 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc3c5da916694a38a26b1ac168048634)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.329296 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.376240 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc3c5da916694a38a26b1ac168048634', 'request_id': 'req-d7cf21ab-1512-4bcd-a96b-d6f28a299657'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:47.384423 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc3c5da916694a38a26b1ac168048634', 'request_id': 'req-d7cf21ab-1512-4bcd-a96b-d6f28a299657'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:47.390564 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc3c5da916694a38a26b1ac168048634', 'request_id': 'req-d7cf21ab-1512-4bcd-a96b-d6f28a299657'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:47.400495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] The token cache is being invalidate because project cc3c5da916694a38a26b1ac168048634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:37:47.400777 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:47.401392 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-d7cf21ab-1512-4bcd-a96b-d6f28a299657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:47.402879 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1482/2950] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:37:47 2022] DELETE /identity/v3/projects/cc3c5da916694a38a26b1ac168048634 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:47.407678 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671daca-d7e1-4a21-bd97-d6df1fcd6625 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.407952 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671daca-d7e1-4a21-bd97-d6df1fcd6625 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.408226 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671daca-d7e1-4a21-bd97-d6df1fcd6625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.427464 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d671daca-d7e1-4a21-bd97-d6df1fcd6625 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:47.428628 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d671daca-d7e1-4a21-bd97-d6df1fcd6625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:47.446005 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:47.489743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1469/2951] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:47.497468 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe78138-7716-45b5-9175-01a492e74497 None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.497757 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe78138-7716-45b5-9175-01a492e74497 None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.498072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe78138-7716-45b5-9175-01a492e74497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.517032 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-4fe78138-7716-45b5-9175-01a492e74497 None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:47.518316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-4fe78138-7716-45b5-9175-01a492e74497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:47.526010 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1483/2952] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:47.530019 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671daca-d7e1-4a21-bd97-d6df1fcd6625 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.539923 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671daca-d7e1-4a21-bd97-d6df1fcd6625 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.544429 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.545017 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.545299 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.545570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.546250 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:37:47.546558 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.548138 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.559928 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1470/2953] 10.222.0.88 () {58 vars in 1153 bytes} [Mon Sep 19 12:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:37:47.563909 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe78138-7716-45b5-9175-01a492e74497 None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.572837 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe78138-7716-45b5-9175-01a492e74497 None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.577429 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.578049 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.578321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.578603 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] PATH_INFO: `/v3/projects` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.579199 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.581076 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.607823 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1484/2954] 10.222.0.88 () {60 vars in 1153 bytes} [Mon Sep 19 12:37:47 2022] POST /identity/v3/projects => generated 397 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:37:47.612525 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.622489 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f836e74-b18e-42d0-bc22-a3f974baa035 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.677552 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.678191 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.678472 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.678743 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.679411 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.681308 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.702359 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1471/2955] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Sep 19 12:37:47 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:37:47.706514 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.716860 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7990d-662a-43d1-86dd-1874e7489392 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.722016 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.722610 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.722887 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.723161 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.723803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.725384 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.735077 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1485/2956] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:47.739425 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.749228 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355bf89-9456-4e2a-b10d-ac9be3c15e8b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.753747 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a7ac9-536f-48b1-9382-1d81edfc2ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.754360 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7ac9-536f-48b1-9382-1d81edfc2ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.754639 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7ac9-536f-48b1-9382-1d81edfc2ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.754912 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7ac9-536f-48b1-9382-1d81edfc2ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.755545 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a7ac9-536f-48b1-9382-1d81edfc2ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.757119 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a7ac9-536f-48b1-9382-1d81edfc2ba4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.767517 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1472/2957] 10.222.0.88 () {58 vars in 1122 bytes} [Mon Sep 19 12:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:37:47.771357 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.781336 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810897-8566-44ee-9c55-c2587d7fd9f8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.786437 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.787072 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.787351 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.787621 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] PATH_INFO: `/v3/projects/a5c5583060c84ff0b4466363bac6412d/users/a6c2d26617d24d64917329fdd660df2a/roles/3530f0bc598248418cc693afa8f4b150` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.802223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c5583060c84ff0b4466363bac6412d, user_id=a6c2d26617d24d64917329fdd660df2a, role_id=3530f0bc598248418cc693afa8f4b150)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:47.804802 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:47.817420 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1486/2958] 10.222.0.88 () {60 vars in 1483 bytes} [Mon Sep 19 12:37:47 2022] PUT /identity/v3/projects/a5c5583060c84ff0b4466363bac6412d/users/a6c2d26617d24d64917329fdd660df2a/roles/3530f0bc598248418cc693afa8f4b150 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:47.827818 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395c448-1b78-462c-98d8-1c799a2599ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.828110 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395c448-1b78-462c-98d8-1c799a2599ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.828387 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395c448-1b78-462c-98d8-1c799a2599ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:47.856933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0395c448-1b78-462c-98d8-1c799a2599ad None None] MFA Rules not processed for user `a6c2d26617d24d64917329fdd660df2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:47.858219 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0395c448-1b78-462c-98d8-1c799a2599ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:47.920706 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1473/2959] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:47.932321 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:47.934071 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:47.955895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:47.993753 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0484e-2bbe-433d-a44c-575f45c9e1c2 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:47.997381 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37508f3d-137d-4662-bf2e-1be411f142f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37099ac7c39948f0afd272a2b41949ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:47.998009 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37508f3d-137d-4662-bf2e-1be411f142f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:47.998289 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37508f3d-137d-4662-bf2e-1be411f142f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:47.998563 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37508f3d-137d-4662-bf2e-1be411f142f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:48.007261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37508f3d-137d-4662-bf2e-1be411f142f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:48.008606 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37508f3d-137d-4662-bf2e-1be411f142f8 service cinder] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:48.017817 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1487/2960] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:56.071882 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395c448-1b78-462c-98d8-1c799a2599ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:56.090894 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:37:56.133897 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395c448-1b78-462c-98d8-1c799a2599ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:56.138315 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:56.138946 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:56.139224 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:56.139495 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] PATH_INFO: `/v3/users/a6c2d26617d24d64917329fdd660df2a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:56.140593 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c2d26617d24d64917329fdd660df2a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:56.143028 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:56.189655 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c2d26617d24d64917329fdd660df2a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.190320 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c2d26617d24d64917329fdd660df2a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.197142 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c2d26617d24d64917329fdd660df2a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.203931 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c2d26617d24d64917329fdd660df2a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.216200 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c2d26617d24d64917329fdd660df2a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.222064 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1474/2961] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:37:56 2022] DELETE /identity/v3/users/a6c2d26617d24d64917329fdd660df2a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:37:56.226922 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:56.227418 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:56.227538 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:56.247772 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] MFA Rules not processed for user `e4f47c952a6a40bf944493656ebcef14`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:37:56.248989 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:56.307662 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1488/2962] 10.222.0.88 () {58 vars in 958 bytes} [Mon Sep 19 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:37:56.317891 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:56.319294 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:56.374805 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1a0bf-c99f-4d4a-9dd2-4c4a95fd3c36 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:56.378970 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:56.379570 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:56.379856 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:56.380131 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:56.388990 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:56.390467 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:56.399728 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1475/2963] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:37:56.668800 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:37:56.670551 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:37:56.685080 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5a6f6-4953-451a-b953-aa0643d59dbe None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:37:56.689698 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:37:56.690376 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:37:56.690638 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:37:56.691554 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] PATH_INFO: `/v3/projects/a5c5583060c84ff0b4466363bac6412d` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:37:56.692985 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c5583060c84ff0b4466363bac6412d)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:37:56.694780 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:37:56.741895 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c5583060c84ff0b4466363bac6412d', 'request_id': 'req-067ade0f-6e2f-4542-a80c-e1c13c294cf8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.749133 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c5583060c84ff0b4466363bac6412d', 'request_id': 'req-067ade0f-6e2f-4542-a80c-e1c13c294cf8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.755451 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c5583060c84ff0b4466363bac6412d', 'request_id': 'req-067ade0f-6e2f-4542-a80c-e1c13c294cf8'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.765457 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] The token cache is being invalidate because project a5c5583060c84ff0b4466363bac6412d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:37:56.765744 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.766395 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:37:56.767902 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1489/2964] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:37:56 2022] DELETE /identity/v3/projects/a5c5583060c84ff0b4466363bac6412d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:38:06.395594 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:38:06.762608 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:39:18.312790 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:39:18.331896 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92472-3009-4abb-a44b-24538fec1e9f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:39:18.378310 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:39:18.378922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:18.379195 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:18.379459 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:18.381160 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:18.465934 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:39:18.467266 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:39:18.486460 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1476/2965] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:39:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:39:21.598677 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:39:21.600244 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:21.612803 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ade0f-6e2f-4542-a80c-e1c13c294cf8 admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:39:21.616341 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d523e74-be2b-45b3-8184-db90253b8c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:39:21.616933 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d523e74-be2b-45b3-8184-db90253b8c22 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:21.617214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d523e74-be2b-45b3-8184-db90253b8c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:21.617484 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d523e74-be2b-45b3-8184-db90253b8c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:21.619422 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1d523e74-be2b-45b3-8184-db90253b8c22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:21.629364 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d523e74-be2b-45b3-8184-db90253b8c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:39:21.630828 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d523e74-be2b-45b3-8184-db90253b8c22 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:39:21.639023 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1490/2966] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:39:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:39:25.701274 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:39:25.703313 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:25.714301 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263651e-ccad-4e84-9e34-d07047908b60 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:39:25.717933 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cd4dc-3219-4576-b2bf-0bbbd2e808d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:39:25.718532 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cd4dc-3219-4576-b2bf-0bbbd2e808d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:25.718814 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cd4dc-3219-4576-b2bf-0bbbd2e808d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:25.719086 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cd4dc-3219-4576-b2bf-0bbbd2e808d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:25.720738 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0b4cd4dc-3219-4576-b2bf-0bbbd2e808d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:25.730113 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cd4dc-3219-4576-b2bf-0bbbd2e808d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:39:25.731510 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cd4dc-3219-4576-b2bf-0bbbd2e808d7 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:39:25.739491 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1477/2967] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:39:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:39:26.867694 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154427-aac8-43f2-a681-04c9cb65ce5b None None] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:26.868261 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154427-aac8-43f2-a681-04c9cb65ce5b None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:26.868368 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154427-aac8-43f2-a681-04c9cb65ce5b None None] PATH_INFO: `/` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:26.869891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1491/2968] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 12:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 12:39:26.875613 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0c2fe-744a-4c86-99cb-1718bd6a25cb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:26.876046 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0c2fe-744a-4c86-99cb-1718bd6a25cb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:26.876282 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0c2fe-744a-4c86-99cb-1718bd6a25cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:26.924978 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0da0c2fe-744a-4c86-99cb-1718bd6a25cb None None] MFA Rules not processed for user `dee035c97cb6447382365faf6b6aca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:39:26.926350 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0da0c2fe-744a-4c86-99cb-1718bd6a25cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:26.933001 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1478/2969] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:39:26.945231 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be154427-aac8-43f2-a681-04c9cb65ce5b None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:39:26.957186 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be154427-aac8-43f2-a681-04c9cb65ce5b None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:39:26.961052 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d67b8-2ba7-4346-97af-de947fc11147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:39:26.961643 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d67b8-2ba7-4346-97af-de947fc11147 service neutron] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:26.961965 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d67b8-2ba7-4346-97af-de947fc11147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:26.962245 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d67b8-2ba7-4346-97af-de947fc11147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:26.970180 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d67b8-2ba7-4346-97af-de947fc11147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:39:26.971590 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d67b8-2ba7-4346-97af-de947fc11147 service neutron] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:39:26.979499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1492/2970] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:39:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:39:27.063496 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474688c-4952-440d-8c0a-f14f8c273240 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:27.063789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474688c-4952-440d-8c0a-f14f8c273240 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:27.064068 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474688c-4952-440d-8c0a-f14f8c273240 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:27.065410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1479/2971] 10.222.0.88 () {54 vars in 921 bytes} [Mon Sep 19 12:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 12:39:27.070794 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] REQUEST_METHOD: `POST` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:27.071083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:27.071359 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:27.110758 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.auth.core [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] MFA Rules not processed for user `345c823f8d85402ba3b07f39ed5e2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:39:27.111978 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:27.121280 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1493/2972] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Sep 19 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:39:27.132830 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474688c-4952-440d-8c0a-f14f8c273240 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:39:27.144293 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474688c-4952-440d-8c0a-f14f8c273240 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:39:27.148210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee035c97cb6447382365faf6b6aca53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:39:27.148642 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:27.148922 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:27.149194 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:27.160290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:39:27.161698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:39:27.172344 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1480/2973] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:39:27.236106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:39:27.237725 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:27.311046 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24723ca3-78ee-40a8-a3b4-d3d6f31a153f None None] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:39:27.317004 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:39:27.317619 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:27.318128 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:27.318412 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:27.320570 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:27.326914 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:39:27.329065 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:27.334689 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:39:27.337332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:39:27.346836 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcfc0e-7d4e-477c-ae09-59da50b4843a service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:39:27.350214 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1494/2974] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:39:27.352902 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '345c823f8d85402ba3b07f39ed5e2fca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:39:27.353437 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:39:27.353787 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:39:27.354045 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:39:27.355907 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:39:27.372038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:39:27.374835 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:39:27.386049 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1481/2975] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:39:37.344263 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=53 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:39:37.381255 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:41:01.434036 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:41:01.435971 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:41:01.466256 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d46d0-a569-4c7d-82f3-355364203405 service placement] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:41:01.471937 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:41:01.472891 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:41:01.473171 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:41:01.473443 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:41:01.475318 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:41:01.487938 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:41:01.489719 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:41:01.500288 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1495/2976] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:41:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:41:05.796192 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:41:05.807458 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9065bc-4338-4b17-97cd-4e0f523f3849 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:41:05.812692 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:41:05.813287 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:41:05.813704 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:41:05.814009 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:41:05.815889 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:41:05.827268 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:41:05.828825 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:41:05.840410 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1482/2977] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:41:08.647262 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:41:08.665483 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab333f79-be34-43ea-bc4e-994d341570f0 service placement] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:41:08.671247 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:41:08.672243 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] REQUEST_METHOD: `GET` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:41:08.672768 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:41:08.673232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:41:08.675847 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:41:08.680409 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:41:08.682433 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:41:08.690316 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:41:08.691955 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:41:08.705332 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1496/2978] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:41:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:41:08.757210 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8671bf-768b-40ee-a846-06f21dc7582a service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:41:08.761698 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:41:08.762356 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:41:08.762641 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:41:08.762918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] PATH_INFO: `/v3/users/49c6e6d0e00449eaabc76b1211ec8c43` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:41:08.764023 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c6e6d0e00449eaabc76b1211ec8c43)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:41:08.765972 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:41:08.813257 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c6e6d0e00449eaabc76b1211ec8c43'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:08.813980 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c6e6d0e00449eaabc76b1211ec8c43'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:08.820898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c6e6d0e00449eaabc76b1211ec8c43'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:08.830031 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c6e6d0e00449eaabc76b1211ec8c43'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:08.842198 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c6e6d0e00449eaabc76b1211ec8c43'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:08.848797 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1483/2979] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:41:08 2022] DELETE /identity/v3/users/49c6e6d0e00449eaabc76b1211ec8c43 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:41:09.018836 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:41:09.029283 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd654a9-8b16-474d-a1dc-ba6f68781795 service placement] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:41:09.086877 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:41:09.087497 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:41:09.087773 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:41:09.088039 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] PATH_INFO: `/v3/projects/84a8601fd7bf4667a9b6901f6b4147cf` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:41:09.089167 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a8601fd7bf4667a9b6901f6b4147cf)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:41:09.091083 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:41:09.136428 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a8601fd7bf4667a9b6901f6b4147cf', 'request_id': 'req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.143302 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a8601fd7bf4667a9b6901f6b4147cf', 'request_id': 'req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.149941 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a8601fd7bf4667a9b6901f6b4147cf', 'request_id': 'req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.160721 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] The token cache is being invalidate because project 84a8601fd7bf4667a9b6901f6b4147cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:41:09.161008 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.161644 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.163150 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1497/2980] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:41:09 2022] DELETE /identity/v3/projects/84a8601fd7bf4667a9b6901f6b4147cf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:41:09.167290 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:41:09.168996 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:41:09.183118 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b7624-401f-4c7c-ab0a-db601aedd1ac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:41:09.238634 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:41:09.239190 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:41:09.239474 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:41:09.239768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] PATH_INFO: `/v3/users/ea7915e975f946b0b8d854b02b96da77` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:41:09.240955 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7915e975f946b0b8d854b02b96da77)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:41:09.242918 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:41:09.248238 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:41:09.288397 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7915e975f946b0b8d854b02b96da77'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.289022 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7915e975f946b0b8d854b02b96da77'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.296404 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7915e975f946b0b8d854b02b96da77'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.304424 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7915e975f946b0b8d854b02b96da77'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.316234 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7915e975f946b0b8d854b02b96da77'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.322029 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1484/2981] 10.222.0.88 () {58 vars in 1224 bytes} [Mon Sep 19 12:41:09 2022] DELETE /identity/v3/users/ea7915e975f946b0b8d854b02b96da77 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:41:09.494904 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Authenticating user token {{(pid=81405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:41:09.496327 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.fernet_utils [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:41:09.515000 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70e7fd-62b3-4c3f-bf13-41ebb2ae498f admin admin] Validating token access rules against request {{(pid=81405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:41:09.566408 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:41:09.576495 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4f47c952a6a40bf944493656ebcef14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a73ceb365cbe45598f5feb33f3998086', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:41:09.577129 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:41:09.577410 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] SCRIPT_NAME: `/identity` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:41:09.577687 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] PATH_INFO: `/v3/projects/d342593fce454ca6a9d9890916191dcb` {{(pid=81405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:41:09.578843 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d342593fce454ca6a9d9890916191dcb)` {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:41:09.581232 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] RBAC: Authorization granted {{(pid=81405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:41:09.625869 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd342593fce454ca6a9d9890916191dcb', 'request_id': 'req-faa0de86-a933-4581-b378-64c36303880c'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.632956 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd342593fce454ca6a9d9890916191dcb', 'request_id': 'req-faa0de86-a933-4581-b378-64c36303880c'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.639223 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd342593fce454ca6a9d9890916191dcb', 'request_id': 'req-faa0de86-a933-4581-b378-64c36303880c'} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.650499 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] The token cache is being invalidate because project d342593fce454ca6a9d9890916191dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:41:09.651106 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.651455 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG keystone.notifications [None req-faa0de86-a933-4581-b378-64c36303880c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:41:09.652926 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: [pid: 81405|app: 0|req: 1498/2982] 10.222.0.88 () {58 vars in 1233 bytes} [Mon Sep 19 12:41:09 2022] DELETE /identity/v3/projects/d342593fce454ca6a9d9890916191dcb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:41:19.318592 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:41:19.647248 dv-cinder-858293-1-iscsi devstack@keystone.service[81405]: DEBUG dbcounter [-] [81405] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:46:07.806759 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:46:07.808170 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:46:07.916898 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e2232-4585-4a2f-b323-7b58b1304323 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:46:07.974890 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518daf1-e9f1-4e55-86f5-0eb9e3d71bac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38726440e2b444d9e2ccbb8681b9ffe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb8af98c7bf40ca8e628a1d04083399', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:46:07.975488 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518daf1-e9f1-4e55-86f5-0eb9e3d71bac service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:46:07.975768 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518daf1-e9f1-4e55-86f5-0eb9e3d71bac service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:46:07.976038 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518daf1-e9f1-4e55-86f5-0eb9e3d71bac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:46:07.977789 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e518daf1-e9f1-4e55-86f5-0eb9e3d71bac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:46:07.989177 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518daf1-e9f1-4e55-86f5-0eb9e3d71bac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:46:07.990783 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518daf1-e9f1-4e55-86f5-0eb9e3d71bac service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:46:08.000901 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1485/2983] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Sep 19 12:46:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:46:17.996515 dv-cinder-858293-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=73 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}